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This  supplement  describes  the  rationale  and  methodology  supporting  the  risk 
management  criteria  defined  in  RCC  323-99  Range  Safety  Criteria  for  Unmanned  Air  Vehicles. 
It  provides  amplifying  background  information,  examples,  definitions,  and  alternatives  to 
consider  when  establishing  UAV  risk  management.  The  rationale  descriptions  contained  in  the 
supplement  are  organized  to  correspond  paragraph  by  paragraph  to  the  criteria  document. 


Multiple  criteria  are  used  to  examine  flight  safety  from  the  perspective  to  ensure  a 
thorough  review.  Different  viewpoints  reduce  the  risk  of  unrecognized  hazards  and  help  to 
quickly  identify  and  isolate  deficiencies.  The  criteria  are  used  to  break  up  the  safe  to  fly? 
question  into  a  series  of  presuppositions: 

a.  Are  system  hazards  recognized  and  risk  controls  available? 

1 .  Risk  management  criteria 

b.  How  is  this  range  vulnerable  to  these  identified  system  hazards? 

2.  Casualty  expectation  criteria 

3.  Property  damage  criteria 

4.  Midair  collision  avoidance  criteria 

c.  If  safeguards  are  needed  to  reduce  risk,  will  they  work? 

5.  Adequacy  of  safeguards  criteria 

This  supplement  is  based  on  guidance  from  safety  specialists,  existing  reference  standards 
and  policies,  and  established  procedures  from  ranges  that  routinely  support  UAV  operations. 

Final  authority  to  conduct  a  test  or  operation  on  a  range  rests  with  the  Range  Commander 
or  his  or  her  designated  representative.  RCC  323-99  provides  definitive  criteria  for  making  this 
risk  decision.  Definitive  criteria  which  has  been  reviewed  and  approved  by  the  Range 
Commanders  Council  provides  a  standard  by  which  the  Range  Commanders  actions  can  be 
compared  to  best  practice  and  to  what  a  reasonable  person  would  do  in  similar  circumstances. 

The  technology  and  performance  limits  of  unmanned  air  vehicles  continue  to  progress  at 
a  rapid  pace;  the  corresponding  range  safety  methods,  standards,  and  procedures  must  keep  up 
with  these  changes.  This  supplement  describes  best  practices  and  procedures  known  at  the  time 
of  its  publication.  The  supplement  is  considered  a  living  document  and  will  be  updated 
regularly. 

Change  recommendations  are  encouraged  and  appreciated,  and  should  be  forwarded  to 
rcc@wsmr.army.mil. 
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ACRONYMS 


ADS-B 

Automatic  Dependent  Surveillance  -  Broadcast 

AR 

Army  Regulation 

AFB 

Air  Force  Base 

AFI 

Air  Force  Instruction 

AFP  AM 

Air  Force  Pamphlet 

ATC 

Air  Traffic  Control 

AWACS 

Airborne  Warning  and  Control  System 

CFR 

Code  of  Federal  Regulations 

COA 

Certificate  of  Authorization 

DB 

Decibel 

DOD 

Department  of  Defense 

DR 

Dead  Reckoning 

EWR 

Eastern  and  Western  Test  Range 

FAA 

Federal  Aviation  Administration 

FAR 

Federal  Aviation  Regulations 

FMECA 

Failure  Modes,  Effects  and  Criticality  Analysis 

FTA 

Fault  Tree  Analysis 

FTS 

Flight  Termination  System 

GCS 

Ground  Control  Station 

GPS 

Global  Positioning  System 

GSFC 

Goddard  Space  Flight  Center 

IEC 

International  Electrotechnical  Committee 

IFF 

Identification  Friend  or  Foe 

IFR 

Instrument  Flight  Rules 

IMC 

Instrument  Meteorological  Conditions 

INS 

Inertial  Navigation  System 

MARSA 

Military  Assumes  Responsibility  for  Separation  of  Aircraft 

MRTFB 

Master  Range  Test  Facility  Base 

MRU 

Military  Radar  Unit 

MTBF 

Mean  Time  Between  Failure 

NASA 

National  Aeronautic  and  Space  Administration 

NATO 

North  Atlantic  Treaty  Organization 

NATOPS 

Naval  Aviation  Training  and  Operating  Procedures  Standardization 

NOAA 

National  Oceanographic  and  Atmospheric  Administration 

NHB 

NASA  Handbook 

ORM 

Operations  Risk  Management 

RCC 

Range  Commanders  Council 

RDT&E 

Research  Development  Test  and  Evaluation 

RF 

Radio  Frequency 

RFI 

Radio  Frequency  Interference 

RLV 

Re-usable  Launch  Vehicle 

ROA 

Remotely  Operated  Aircraft 

RPV 

Remotely  Piloted  Vehicle 

IX 


SATCOM 

Satellite  Communications 

SOP 

Standard  Operating  Procedure 

STANAG 

Standardization  Agreement  (NATO) 

TCAS 

Traffic  Alert  and  Collision  Avoidance  System 

UAV 

Unmanned  Air  Vehicle  or  Uninhabited  Air  Vehicle 

UHF 

Ultra  High  Frequency 

VFR 

Visual  Flight  Rules 

YHF 

Very  High  Frequency 

VMC 

Visual  Meteorological  Conditions 

WFF 

Wallops  Flight  Facility 

X 


GLOSSARY 


Acceptable  Risk 

1.  The  portion  of  identified  risk  that  is  allowed  to  persist  without  further  controls.  It  is 
accepted  by  the  appropriate  decision-maker  (AFP AM  91-214).  2.  A  predetermined  criterion  or 
standard  for  a  maximum  risk  ceiling  which  permits  the  evaluation  of  cost,  national  priority 
interests,  and  number  of  tests  to  be  conducted  (RCC  321-00). 

Casualty  Expectation 

Risk  to  people  measured  as  a  function  of  expected  fatalities  per  flight  hour  of  operation. 

Collective  Risk 

The  total  risk  to  an  exposed  population;  the  expected  total  number  of  individuals  who  will 
be  fatalities.  Defined  as  Expected  Fatalities.  Collective  risk  is  specified  as  either  a  per  mission 
or  per  year  value  (RCC  321-00). 

Containment 

The  range  safety  strategy  of  ensuring  risk  is  minimized  by  keeping  hazardous  operations 
within  hazard  areas  verified  to  be  clear  of  vulnerable  personnel  or  property. 

Expected  Fatalities 

The  expected  number  of  individuals  who  will  be  fatalities.  Used  to  define  Collective  Risk. 
This  risk  is  expressed  with  the  following  notation:  IE-7  =  10"7  =  1  in  ten  million  (RCC  321-00). 

Exposure 

The  number  of  persons  or  resources  affected  by  a  given  event,  or  over  time,  repeated 
events.  This  can  be  expressed  in  terms  of  time,  proximity,  volume,  or  repetition.  This  parameter 
may  be  included  in  the  estimation  of  severity  or  probability,  or  included  separately  (AFPAM  91- 
214). 

Fail  safe 

1 .  A  design  feature  that  ensures  the  system  remains  safe,  or  in  the  event  of  failure,  causes 
the  system  to  revert  to  a  state  that  will  not  cause  a  mishap  (MIL-STD-882D)  2.  A  method  built 
into  flight  termination  systems  that  will  activate  an  output  upon  the  loss  of  power  and/or  RF 
signal  and/or  tone.  (RCC-3 19-99) 

Gambling 

Making  risk  decisions  without  reasonable  or  prudent  assessment  or  management  of  the 
risks  involved  (AFPAM  91-214). 


Hazard 

Any  real  or  potential  condition  that  can  cause  mission  degradation,  injury,  illness,  or  death 
to  personnel  or  damage  to  or  loss  of  equipment  or  property  (AFP AM  91-214). 

Hazard  Area 

A  geographical  or  geometric  surface  area  that  is  susceptible  to  a  hazard  from  a  planned 
event  or  unplanned  malfunction  (RCC  321-00) 

Mishap 

An  unplanned  event  or  series  of  events  resulting  in  death,  injury,  occupational  illness,  or 
damage  to  or  loss  of  equipment  or  property  (AFP  AM  91-214,  MIL-STD-882D). 

Probability 

The  likelihood  that  an  event  will  occur  (AFP AM  91-214). 

Residual  Risk 

The  remaining  risk  that  exists  after  all  mitigation  techniques  have  been  implemented  or 
exhausted  (MIL-STD-882D) 

Risk 

An  expression  of  mishap  consequences  in  terms  of  probability  of  an  event  occurring,  the 
severity  of  the  event  and  the  exposure  of  personnel  or  resources  to  potential  loss  or  harm 
(AFPAM  91-214). 

Safeguard 

Hardware  component,  software  routine,  operator  procedure,  or  some  combination  intended 
to  mitigate  risks. 

Safety  Critical 

Any  condition,  event,  operation,  process,  or  item  whose  proper  recognition,  control, 
performance,  or  tolerance  is  essential  to  safe  system  operation  and  support  (MIL-STD-882D) 

Severity 

The  expected  consequences  of  an  event  in  terms  of  degree  of  impact  on  the  mission,  injury, 
or  damage  (AFPAM  91-214). 

Waiver 

Granted  use  or  acceptance  of  an  article  that  does  not  meet  the  specified  requirement  (RCC 
319-99) 
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1.  HAZARD  RECOGNITION  AND  RISK  REDUCTION  CRITERIA 

In  RCC  Document  323-99,  Range  Safety  Criteria  for  Unmanned  Air  Vehicles,  five  separate 
criteria  are  used  to  determine  if  a  UAV  is  safe  to  fly  on  a  particular  range.  The  first  criteria  (risk 
management)  address  the  question  “Are  system  hazards  recognized  and  risk  controls  available?’ 


1.0.1  Risk  Management. 

Risk  management  is  a  process  used  by  decision-makers  to  handle  potentially  hazardous 
operations.  The  objective  of  the  risk  management  process  is  to  ensure  hazards  are  identified, 
evaluated  and  eliminated  or  to  ensure  that  the  associated  risks  are  reduced  to  an  acceptable  level. 
“Risk  Management  Criteria,”  as  stated  in  document  323-99,  is  a  tool  that  can  be  used  to  create  or 
review  a  UAV  risk  management  program  to  ensure  range  safety  criteria  is  met. 

1.0.2  Why  Risk  Management  is  Required. 

1.0.2.1  References.  Risk  management  is  a  requirement  of  the  Department  of  Defense  (DOD) 
and  the  National  Aeronautics  Space  Administration  (NASA).  Use  of  Operational  Risk 
Management  (ORM)  (i.e.,  hazard  analysis,  risk  reduction,  and  implementation  of  risk  controls) 
is  mandatory  throughout  DOD.  References  include  OPNAV  3500.39,  Air  Force  Instruction  91- 
213,  and  Army  AR  385-10.  NASA  also  requires  hazard  analysis  and  risk  controls  for  UAV 
projects.  Applicable  references  include:  NHB  1700.1  (Vl-B)  dated  1993,  NASA  Safety  Policy 
and  Requirements  Document,  and  RSM-93,  Range  Safety  Manual  for  Goddard  Space  Flight 
Center  (GSFC)/Wallops  Flight  Facility  (WFF). 

1.0.2.2  Approach.  Risk  management  is  a  systematic  approach  performed  on  the  complete 
system  and  should  be  integrated  as  early  as  possible  because  risks  are  more  easily  assessed  and 
managed  in  the  planning  stages  of  an  operation.  Risks  may  be  acceptable,  dependent  on  the 
probability,  severity,  and  necessity  to  the  successful  completion  of  the  mission.  With  adequate 
hazard  analysis,  the  range  can  make  informed  decisions  and  apply  the  appropriate  level  of 
restrictions.  An  inadequate  analysis  may  lead  to  overly  restrictive  requirements  on  the  user  or 
unacceptable  risk  to  the  range. 
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1.0.3  The  Risk  Management  Program. 

If  the  user  has  a  risk  management  program  in  place,  document  323-99,  Section  1,  “Risk 
Management  Criteria,”  can  be  used  to  validate  the  approach  and  the  completeness  of  the 
program.  When  the  users’  risk  management  program  meets  these  criteria,  additional  analysis  can 
be  avoided,  resulting  in  significant  cost  and  time  savings. 

If  the  user’s  risk  management  program  is  not  adequate,  the  criteria  can  be  used  to  focus  on 
specific  problem  areas.  A  checklist  of  UAV  specific  hazards  is  provided  to  further  assist  the 
analyst  in  determining  if  anything  has  been  missed.  If  the  user’s  risk  management  program  is 
unacceptable  or  non-existent,  the  range  should  require  that  a  risk  management  program  be 
established.  A  checklist  is  provided  as  a  starting  point  for  a  UAV  program  hazard  review. 

Note;  The  risk  management  criteria  is  intended  to  assess  the  approach  and  completeness  of  the 
range  users’  risk  management  program,  not  to  mandate  the  format. 

Appendix  A  provides  a  list  of  references  and  information  sources  that  describe  general 
methods  to  implement  a  risk  management  process  in  range  operations.  This  document  will 
support  those  risk  management  processes  that  are  specific  to  the  UAV  range  test  and  operations 
mission.  Figure  1. 0.3-1  diagrams  the  concepts  of  the  risk  management  process  that  are  discussed 
in  the  following  sections. 
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FIGURE  1.0. 3-1.  The  Risk  Management  Process 
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1.1  Hazards  Identified.  The  hazards  associated  with  the  proposed  UAV  operations  have  been 
explicitly  stated,  based  on  lessons  learned  and  hazard  analysis.  Vulnerability  to  unidentified  risk 
is  reduced  through  hazard  analysis  efforts. 

Both  the  range  and  the  user  must  have  a  technical  and  operational  understanding  of 
potential  UAV  system  hazards  to  operate  safely.  This  information  also  enables  safety  personnel 
to  identify  potential  system  hazards  and  review  the  existing  hazard  controls.  Without  explicitly 
identifying  system  hazards,  the  range  is  vulnerable  to  hazards  that  may  be  present  but  are  not 
recognized. 

Hazards  associated  with  the  proposed  UAV  operation  can  be  identified  based  on  system 
knowledge,  hazard  analysis,  past  experience,  and  lessons  learned.  The  format  used  to  identify 
the  hazards  is  not  critical,  only  that  the  hazards  be  clearly  identified.  Examples  of  documents 
that  may  identify  hazards  include  hazard  lists,  hazard  analyses,  and  user  manuals. 

Tables  1.1-1  through  1.1-5  list  generic  hazard  conditions  and  vehicle  failure  modes  which 
can  lead  to  loss  of  the  UAV,  a  midair  collision,  serious  injury,  and/or  death.  The  background 
information  summarized  in  these  tables  is  based  on  mishap  data  as  well  as  UAV  hazard  analyses. 
These  tables  are  generic,  not  all-inclusive,  and  may  or  may  not  apply  to  a  specific  vehicle  or 
situation. 
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Table  1.1-1  lists  hazardous  conditions  that  may  result  in  loss  of  control  of  the  UAV, 
which  can  ultimately  result  in  an  uncontrolled  crash  or  collision. 

TABLE  1.1-1.  HAZARDOUS  CONDITIONS  WHICH  MAY  RESULT  IN 

UNCONTROLLED  FLIGHT 


Hazardous  condition 

Cause 

Loss  of  propulsion 

•  engine  failure 

•  fuel  starvation 

•  stuck  throttle 

•  icing  /  weather 

Loss  of  lift 

•  structural  failure 

•  icing  /  weather 

Loss  of  heading  /  attitude  /  position 
information 

•  heading  /  attitude  system  failure 

•  navigation  system  failure 

Unplanned  loss  of  link 

•  radio  frequency  interference 

•  flight  beyond  horizon 

•  antenna  masking 

•  loss  of  ground  control  station 

•  software  interrupt  between  ground 
control  station  and  air  vehicle 

•  atmospheric  attenuation 

•  inadvertent  deactivation  of  autopilot 

•  loss  of  satellite  link 

Loss  of  control  surface  performance 

•  stuck  servo 

•  autopilot  failure 

•  icing  /  damage  to  control  surface 

Loss  of  UAV  electrical  power 

•  generator  failure 

•  backup  battery  failure 

•  excessive  load  from  payload 

Loss  of  ground  control  station  (GCS) 

•  Loss  of  GCS  power 

•  GCS  transmitter/  receiver  /  antenna 

failure 

•  GCS  computer  failure 
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Some  mishaps  occur  when  the  vehicle  impacts  the  ground  even  though  the  vehicle  is  still 
capable  of  controlled  flight.  This  category  of  mishap  is  referred  to  as  “controlled  flight  into 
terrain.”  Hazardous  conditions  and  corresponding  causes  related  to  “controlled  flight  into 
terrain”  are  listed  in  table  1.1-2. 

TABLE  1.1-2.  HAZARDOUS  CONDITIONS  WHICH  MAY  RESULT  IN  CONTROLLED 

FLIGHT  INTO  TERRAIN 


% 


Hazardous  Condition 

Cause 

Mission  planning  error  or  operator  error 

•  flight  below  minimum  enroute  altitude 

•  undetected  man-made  obstacles  (towers, 
cables) 

Altitude  error 

•  incorrect  barometer  setting 

•  inadequate  alert  for  altitude  deviation 

Navigation  error 

•  nav  system  failure 

•  nav  system  discrepancy  (INS  vs.  GPS) 

•  map  display  inaccuracy 

Failure  to  see  and  avoid  terrain 

•  no  capability 

•  autonomous  operation 

Loss  of  link  “fly  home”  mode 

•  mission  planning  error  for  loss  of  link 
mode 

Table  1.1-3  lists  potential  hazardous  conditions  and  causes  related  to  a  mid-air  collision 
with  other  aircraft. 


TABLE  1.1-3.  HAZARDOUS  CONDITIONS  WHICH  MAY  RESULT  IN  MIDAIR 

COLLISION 


Hazardous  condition 

Cause 

Navigation  error 

•  nav  system  failure 

•  nav  system  discrepancy  (INS  vs.  GPS) 

•  map  display  inaccuracy 

Altitude  error 

•  incorrect  barometer  setting 

•  inadequate  alert  for  altitude  deviation 

Unable  to  “see-and-avoid” 

•  limited  capability 

•  autonomous  operation 

Mission  planning  error 

•  inadvertent  flight  into  established  routes 
of  other  aircraft 

Not  seen  by  other  aircraft 

•  strobe  /  position  lights  inadequate  or  fail 

•  IFF  failure 

•  TCAS  failure 

•  ATC/UAV  operator  comm  link  failure 
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Mishaps  during  takeoff  and  landing  are  a  significant  percentage  of  all  UAV  mishaps.  Table 
1.1-4  lists  some  hazardous  conditions  and  causes  related  to  this  category  of  mishap. 

TABLE  1.1-4.  HAZARDS  RESULTING  IN  TAKEOFF/LANDING  MISHAPS 


Hazardous  condition 

Cause 

Pilot  induced  oscillation 

•  system  latency 

Automatic  landing  system  failure 

•  RFI 

•  handoff  errors 

•  missed  approach  procedures 

Operator  error 

•  outside  weather  /  wind  limits 

•  internal  pilot  /  external  pilot  handoff 
errors 

* 


Some  factors  can  contribute  to  or  exacerbate  hazardous  conditions  and  increase  the  chance 
of  a  mishap  given  that  a  hazardous  condition  exists.  Table  1.1-5  lists  some  potential  contributing 
factors  and  their  causes. 

TABLE  1.1-5.  CONTRIBUTING  FACTORS  POTENTIALLY  RESULTING  in  VEHICLE 

LOSS 


Contributing  factor 

Cause 

Inadequate  operator  response 

•  failure  to  recognize  flight  critical 
situation 

•  flight-critical  information  missing, 
erroneous,  or  ambiguous 

•  delays  in  information  flow 

Incorrect  inputs  of  flight  critical  parameters 

•  operator  entry  errors 

Operator  information  overload 

•  tasking  Vs  time  available 

•  sensory  overload  over  time 

Critical  information  unavailable, 
inadequate,  blocked,  etc. 

•  design  dependent 

Latency  of  flight  control  commands 

•  operator  far  removed  from  control  loop 

•  non-deterministic  software 

•  control  link  through  satellite 

Operator  fatigue 

•  inadequate  crew  rest 

•  task  saturation 

•  long  /  boring  mission 

Control  of  multiple  UAVs 

•  workload  issues 

Software  paths  to  unsafe  state 

•  unexpected  reboot 

•  inadequate  software  safety  process 
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The  checklist  in  Appendix  B  can  also  be  used  to  help  determine  if  there  are  any  significant 
omissions  from  the  range  user’s  risk  management  program.  This  list  is  not  intended  to  be  all- 
inclusive  for  all  UAV,  missions/operations,  or  ranges  but  is  provided  as  a  basic  guide  or  starting 
point. 

1*2  Hazards  Assessed.  A  hazard  analysis  must  be  performed  and  documented.  This  document 
shall  include  the  level  of  risk  associated  with  identified  hazards. 

Once  hazards  are  identified  they  should  be  expressed  in  terms  of  severity  and  probability  of 
occurrence.  This  analysis  allows  the  range  and  range  users  to  focus  on  hazards  which  are  critical 
and  devote  less  attention  to  those  that  are  clearly  insignificant.  The  range  may  justify  accepting 
some  risks  without  controls  if  the  severity  is  low,  the  probability  is  negligible,  or  the  Range 
Commander  determines  the  benefits  outweigh  the  costs.  If  hazards  are  not  assessed  in  terms  of 
risk  (severity  and  probability),  unnecessary  requirements  may  be  placed  upon  the  user  or  the 
range  may  accept  undue  risk. 

Severity  assessment  should  be  based  on  the  worst  credible  outcome  that  can  be  reasonably 
expected.  For  range  safety  purposes,  the  severity  of  the  hazard  should  be  determined  by  its 
potential  impact  on  people,  property,  and  the  environment.  Measures  of  severity  for  program 
management  can  also  consider  system  loss  and  degradation  or  mission  loss.  Severity  categories 
are  defined  to  provide  a  qualitative  measure  of  the  hazards  severity.  Table  1.2-1  lists  common 
definitions  for  severity  categories. 

TABLE  1.2-1  HAZARD  SEVERITY  CATEGORIES 


Description 

Level 

Effect  on  people 

Effect  on 
property 

Environmental 

effects 

Catastrophic 

I 

death,  permanent 
disability 

greater  than  $1 
million 

severe 

Critical 

II 

severe  injury, 
permanent  partial 
disability, 

hospitalization  for  5 
or  more  people 

$200,000  to  $1 
million 

major 

Marginal 

III 

minor  injury,  1  or 
more  lost  workdays 

$10,000  to 
$200,000 

minor 

Negligible 

IV 

less  than  minor  injury 

less  than  $10,000 

less  than  minor 

A  probability  must  be  assigned  to  each  identified  cause  of  a  hazard.  A  qualitative 
probability  may  be  assigned  early  in  the  mission  planning  stages  and  can  be  combined  with  the 
severity  category  to  determine  an  initial  risk  assessment.  The  Risk  Assessment  Matrix  in  Figure 
1 .2-3  may  be  used  to  prioritize  resources  to  evaluate  and  resolve  hazards.  The  following  are 
generally  accepted  definitions  for  probability. 
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TABLE  1.2-2.  HAZARD  PROBABILITY  LEVELS 


Description 

Level 

Incidents  per 
100,000  flight 
hours  (  note  1) 

Individual  exposure 
rate 

Fleet  or 
inventory 
exposure  rate 

Frequent 

A 

100  or  more 

Likely  to  occur 
frequently 

Continuously 

experienced 

Probable 

B 

10  to  99 

Will  occur  several  times 
in  the  life  of  an  item 

Will  occur 
frequently 

Occasional 

C 

1  to  9.9 

Likely  to  occur  sometime 
in  the  life  of  an  item 

Will  occur 
several  times 

Remote 

D 

0.1  to  0.99 

Unlikely  but  possible  to 
occur  in  the  life  of  an 
item 

Unlikely  but  can 
reasonably  be 
expected  to  occur 

Improbable 

E 

less  than  0.1 

So  unlikely,  it  can  be 
assumed  occurrence  will 
not  be  experienced 

Unlikely  to 
occur,  but 
possible 

Note  1 :  Probability  per  flight  hour  categories  from  NAVAIRINST  5100.1 1 


Hazard  Categorization 

SEVERITY 

CATASTROPHIC 

ii 

CRITICAL 

mp| 

F 

1  p 

R 

spi!  ii 

O 

1  s«*"* JS&L  ] 

U 

E 

N 

C 

Y 

(C)  OCCASIONAL 
1.0-9.9/100K  fit  lm 

(D)  REMOTE 

0.1-0. 99/1 OOK  fit  hrs 

(E)  IMPROBABLE 

■  or  <  0.1/100K  fit  hrs 

. .. .  *. . . 

1-5  High  Safety 
Risk 


6-10  Medium  Safety 
Risk 


11-17  Low  Safety 
Risk 


18-20  Very  Low  Safety 
Risk 


Figure  1.2-3.  Risk  assessment  matrix. 
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1.3  Control  Measures  and  Risk  Decisions.  Control  measures  to  reduce  risks  to  an  acceptable 
level  are  identified. 

Risks  that  are  unacceptable  in  terms  of  severity  and/or  probability  need  to  be  controlled. 

The  user  must  help  identify  specific  strategies,  tools,  or  safeguards  to  eliminate  or  reduce  the  risk 
to  a  level  acceptable  to  the  range. 

According  to  MIL-STD-882,  the  desired  order  of  precedence  for  implementing  control 
measures  is  as  follows: 

•  Design  for  minimum  risk.  Eliminate  the  hazard. 

•  Incorporate  safety  devices. 

•  Provide  warning  devices. 

•  Develop  procedures  and  training. 

1.3.1  Design  for  Minimum  Risk. 

The  best  way  to  control  a  hazard  is  to  eliminate  it  by  changing  the  design  or  adjusting  the 
test  and/or  training  requirements.  If  the  hazard  cannot  be  eliminated,  design  changes  may  reduce 
the  risk  to  an  acceptable  level.  Some  examples  of  design  or  requirement  changes,  which  may 
eliminate  or  reduce  risk  include: 

•  Including  a  highly  reliable  engine  in  the  UAV  design  reduces  the  risk  of  loss  of 
propulsion. 

•  Designing  a  series  of  tests  with  a  gradual  buildup  in  risk  reduces  the  chance  of  sudden 
unexpected  catastrophic  failure. 

•  Confining  test  flights  to  an  unpopulated  area  eliminates  risk  to  people  on  the  ground. 

•  Designing  a  low-level  route  that  avoids  populated  areas  reduces  risk  of  ground 
casualties  from  system  failures. 

•  Establishing  policy  to  avoid  icing  conditions  if  the  vehicle  would  be  at  risk  in  such 
conditions  reduces  the  risk  of  icing  induced  loss  of  lift  or  loss  of  propulsion. 

1.3.2  Incorporate  Safety  Devices. 

If  the  hazard  can  not  be  eliminated  through  design  change,  fixed  or  automatic  safety 
devices  should  be  incorporated.  Provisions  for  periodic  functional  checks  for  these  safety 
devices  should  be  instituted.  Examples  of  safety  devices  include: 

•  Back-up  battery  in  case  of  generator  failure 

•  Redundant  communications  link  in  case  of  failure  of  the  primary  link 

•  Software  “fly-home”  routine  in  case  of  lost  link 

•  Independent  flight  termination  systems 
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1.3.3  Provide  Warning  Devices. 

If  the  risk  cannot  be  reduced  adequately  through  design  change  or  use  of  safety  devices, 
warning  devices  that  detect  the  hazardous  condition  and  alert  personnel  of  the  hazard  can  be 
used.  Procedures  for  functional  checks  of  these  warning  devices  should  be  incorporated. 
Examples  of  warning  devices  are: 

•  Engine  performance  safety  data  displays  at  the  ground  control  station  (i.e.,  overtemp 
alert) 

•  Strobe  lights  to  make  the  UAV  easier  to  see 

•  “Low  fuel”  warning  lights 

•  Warning  calls  from  air  traffic  control  when  the  vehicle  is  approaching  other  traffic  or 
hazard/flight  boundaries 

1.3.4  Develop  Procedures  and  Training. 

If  it  is  impractical  to  eliminate  hazards  or  reduce  risk  adequately  through  design  changes  or 
safety  and  warning  devices,  procedures  and  training  can  be  used.  Safety-critical  procedures 
should  be  standardized  and  documented.  Tasks  and  activities  that  are  safety-critical  may  require 
certification  of  personnel  proficiency.  Examples  of  safety-related  procedures  and  training 
include: 

•  Pre-flight  checklists 

•  Published  cautions  and  warnings 

•  Emergency  procedures 

•  Specific  operating  limits 

•  Established  operator  qualification  procedures 

•  Requirements  for  personal  protective  equipment  in  specific  situations  (i.e.,  hearing 
protection). 

Note:  Procedures  and  training  should  not  be  used  as  the  only  risk  reduction  methods  for  high 
risk  hazards. 

1.4  Hazard  Controls.  Control  measures  used  in  the  hazard  analysis  are  incorporated  into 
range  users  test  plan  or  procedure  document. 

The  range  user  must  show  that  identified  control  measures  are  incorporated,  understood, 
and  documented.  If  required,  test  procedures  and  monitoring  of  the  control  measures  must  be 
certified  and  in  place.  If  the  control  measures  are  not  implemented,  or  the  implementation  is  not 
effective  or  sufficient,  the  hazard  is  still  present.  If  hazards  still  exist  after  all  control  measures 
are  in  place,  the  first  step  is  to  re-evaluate  the  hazard  and  control  measures  and  verify  that 
nothing  was  missed  and  no  other  solutions  are  available.  Once  this  process  has  been  established, 
documentation  of  all  hazards,  their  respective  control  measures,  and  any  remaining  risks  and 
recommendations  must  be  presented  to  the  appropriate  level  of  authority  for  a  wavier.  The 
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deciding  authority  will  consider  the  benefits  versus  the  risks  to  decide  whether  a  waiver  will  be 
granted. 

1.5  Supervision.  Follow-up  evaluations  of  the  control  measures  are  planned  in  order  to  ensure 
effectiveness.  Adjustments  will  be  made  before  continuing  with  the  test  or  operation. 

Independent  review  and  approval  of  the  documentation,  hazard  analysis,  hazard  controls, 
and  test  procedures  and  monitoring  must  take  place  prior  to  the  test  or  operation.  This 
monitoring  of  safety  limits  must  take  place  on  a  continuing  basis  for  each  test  and/or  operation. 

1*6  Alternatives  If  the  Risk  Management  Criteria  Are  Not  Met.  If  normal  risk  management 
criteria  are  not  met,  the  following  alternatives  may  be  exercised. 

•  Range  may  re-evaluate  the  hazard  analysis  incorporating  changes  such  as  flight 
parameters,  flight  path,  and  new  information  from  the  user. 

•  Range  may  impose  restriction  to  planned  flight  to  control  identified  risk. 

•  Range  may  require  additional  control  measures  or  safeguards  to  control  identified 
risk. 

•  User  can  request  a  waiver  from  the  Range  Commander. 

•  User  may  not  get  permission  to  fly  on  this  range. 
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2.  CASUALTY  EXPECTATION  CRITERIA 


In  RCC  Document  323-99,  five  separate  criterion  are  used  to  determine  if  a  UAV  is  safe  to 
fly  on  a  particular  range.  The  first  criterion,  risk  management,  addresses  the  question  Are 
system  hazards  recognized  and  risk  controls  available?”  The  second  criterion,  casualty 
expectation,  looks  at  these  potential  risks  from  the  perspective  of  a  specific  range  and  the 
population,  which  may  be  exposed  to  that  risk.  Casualty  expectation  is  another  measure  of  risk 
that  can  provide  a  basis  for  a  range  commander’s  fly/no  fly  risk  decision.  It  examines  the  risk  to 
people  on  the  ground  from  UAV  operations  being  conducted  overhead. 


Casualty  expectation  is  defined  as  the  collective  risk  or  total  risk  to  an  exposed  population, 
the  total  number  of  individuals  who  will  be  fatalities.  This  criterion  is  met  if  the  hazard  is 
confined  to  unpopulated  areas  (see  par.  2.1  below)  or  if  the  combined  vehicle  reliability  and  the 
population  distribution  beneath  the  planned  route  of  flight  results  in  a  risk  that  is  no  greater  than 
that  for  manned  aircraft  operations  (see  par.  2.2  below). 


2.1  No  Risk  to  Human  Life  Because  Hazard  Is  Contained.  The  planned  route  of  flight  is 
acceptable,  because  the  flight  can  be  confined  to  unpopulated  areas. 

If  the  UAV  is  confined  to  an  unpopulated  area,  there  is  no  risk  of  a  crash  injuring  people  on 
the  ground.  This  approach  is  called  “containment.”  Containment  is  typically  used  for  flight¬ 
testing,  high-risk  operations,  or  if  the  probability  of  vehicle  failure  cannot  be  predicted. 

To  verify  that  potential  hazards  are  adequately  contained,  the  safety  analyst  should  verify 
that  the  area  is  unpopulated,  and  there  are  adequate  control  measures  on  the  vehicle  to  ensure  it 
does  not  leave  the  range.  Verification  that  the  area  is  unpopulated  is  typically  done  by  physically 
patrolling  the  range  or  monitoring  it  remotely  with  video.  Containment  can  be  also 
accomplished  by  erecting  a  barrier  such  as  a  fence. 

The  safety  analyst  should  also  determine  if  the  vehicle  is  able  to  leave  the  range.  For 
instance,  is  the  vehicle’s  maximum  range  greater  than  the  distance  to  the  edge  of  the  unpopulated 
hazard  area?  Are  there  failure  modes  such  as  “lost  link”  or  “stuck  servo”  which  could  result  in 
the  UAV  leaving  a  safe  area?  The  safety  analyst  should  review  the  history  of  the  vehicle  or 
similar  designs  encountering  these  failure  modes  before  determining  if  additional  controls  are 
required. 

If  necessary,  an  independent  or  highly  reliable  system,  e.g.,  Flight  Termination  System 
(FTS),  may  be  required  to  ensure  the  vehicle  does  not  leave  assigned  airspace  above  the 
unpopulated  hazard  area.  If  a  "fly  home"  or  "emergency  mission"  software  routine  is  used  to 
keep  the  vehicle  inside  the  assigned  airspace,  the  evidence  of  software  reliability  must  be 
reviewed.  Chapter  5  discusses  these  review  procedures. 
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System  maturity  may  or  may  not  support  requirements  for  additional  safeguards  to  keep  the 
UAV  inside  assigned  airspace.  A  mature  system  with  a  history  of  many  mishaps  should 
certainly  be  treated  differently  than  a  mature  system  with  few  mishaps. 

2.2  Equivalent  Risk  to  Manned  Aircraft.  A  prediction  of  the  average  risk  to  people  within  the 
planned  area  of flight  or  along  the  planned  route  of flight  is  acceptable,  and  avoidance  of  high 
population  density  "hot  spots"  is  considered. 

Casualty  expectation  provides  an  alternative  to  containment  as  a  basis  for  making  risk 
exposure  decisions. 

RCC  Standard  321-00,  Common  Risk  Criteria  for  National  Test  Ranges,  provides  the 
following  policy  guidance  regarding  the  average  risk  to  people  (i.e.,  casualty  expectation)  as  a 
risk  management  alternative  to  containment: 

“As  a  general  policy,  safety  will  be  maximized  consistent  with  operational  requirements. 

All  ranges  strive  to  achieve  complete  containment  of  debris  resulting  from  normal  and 
malfunctioning  flights.  However,  if  the  planned  mission  cannot  be  accomplished  under  these 
conditions,  a  risk  management  policy  may  be  used  if  authorized  by  the  Range  Commander  or  his 
designated  representative.” 

2.2.1  Casualty  Expectation.  Must  be  less  than  one  casualty  in  a  million  flight  hours. 

One  casualty  in  a  million  flight  hours  is  a  defined  risk  limit  established  by  the  RCC-323 
standard.  This  limit  is  derived  from  risks  related  to  manned  aircraft  as  well  as  system  safety 
precedents.  The  casualty  expectation  approach  to  measuring  risk  is  based  on  the  following 
premises,  which  will  be  amplified  in  this  section: 

•  Acceptable  risk  in  terms  of  casualty  expectation  (fatalities  per  flight  hour)  for  manned 
aircraft  has  been  defined  within  the  system  safety  community. 

•  There  is  regulatory  precedent  that  has  limited  risk  exposure  from  range  operations  to 
the  risk  exposure  comparative  to  overflight  of  manned  aircraft. 

•  The  history  of  risk  exposure  to  people  on  the  ground  from  overflight  by  manned 
aircraft  is  measurable  in  terms  of  casualty  expectation. 

•  Therefore,  defining  a  risk  limit  that  is  consistent  with  system  safety  precedents, 
regulatory  precedents,  and  the  history  of  risk  exposure  to  people  on  the  ground  is 
reasonable. 
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2.2. 1.1  System  Safety  and  Casualty  Expectation. 

Definitions  established  within  the  system  safety  discipline  are  consistent  with  a  one  in  a 
million”  risk  limit  for  casualty  expectation.  MIL-STD-882D,  Department  of  Defense  Standard 
Practice  for  System  Safety,  describes  “High  Risk”  as  the  probability  of  a  fatality  as  occasional 
or  likely  to  occur  in  the  life  of  an  aircraft,  or  likely  to  occur  several  times  in  the  entire  fleet  or 
inventory  of  aircraft.  “Serious  risk”  is  defined  as  the  probability  of  a  fatality  is  remote. 

“Remote”  is  defined  as  unlikely  to  occur  in  the  life  of  a  specific  aircraft,  and  unlikely  but  can 
reasonably  be  expected  to  occur  in  the  entire  fleet  or  inventory  of  aircraft.  Medium  risk  is 
defined  as  the  probability  of  a  fatality  is  “improbable.”  “Improbable”  is  defined  as  so  unlikely, 
it  can  be  assumed  occurrence  may  not  be  experienced  during  the  life  of  a  particular  vehicle,  and 
unlikely  to  occur  but  possible  for  a  fleet  or  large  inventory  of  aircraft. 

NAVAIRINST  5100.1 1  further  defines  risk  exposure  in  terms  of  flight  hours.  It  defines 
“occasional”  as  1  to  9.9  incidents  per  100,000  flight  hours,  and  defines  “remote”  as  0.1  to  0.99 
incidents  per  100,000  flight  hours.  “Improbable”  is  defined  as  less  than  0.1  mishap  per  100,000 
flight  hours. 

2.2.1.2  Regulatory  Precedent. 

Because  overflight  by  manned  aircraft  occurs  on  a  routine  basis,  the  risk  of  overflight  by 
manned  aircraft  is  considered  “acceptable  risk.”  There  is  regulatory  precedent  that  has  limited 
risk  exposure  from  range  operations  to  the  risk  exposure  comparative  to  overflight  of  manned 
aircraft.  According  to  RCC  Document  321-00,  Common  Risk  Criteria  for  National  Test  Ranges: 
Inert  Debris ,  Public  Law  81-60  first  used  this  concept  in  the  establishment  of  the  Air  Force 
Eastern  Test  Range: 

“ Public  Law  (PL)  81-60.  One  precedent  in  U.S.  law  directly  relates  to  the  same  hazard  as 
the  debris  protection  standard:  in  1949,  Congress  enacted  PL  81-60,  Guided  Missiles-Joint  Long 
Range  Proving  Ground,  which  authorized  the  Secretary  of  the  Air  Force  to  establish  a  joint 
proving  ground  at  the  present-day  Eastern  Range  location.  The  law,  however,  only  authorizes 
the  establishment  of  a  range.  Observations  in  legislative  history  delineate  to  a  degree  how  the 
location  must  be  chosen. 

Contained  within  the  language  of  legislative  history  is  the  requirement  for  safe  operation  of 
the  range;  “ From  a  safety  standpoint  [test  flights  of  missiles]  will  be  no  more  dangerous  than 
conventional  airplanes  flying  overhead.  ”  This  language  was  clearly  intended  to  allay  public 
fears  at  the  time  missile  testing  was  in  its  infancy,  and  was  not  intended  to  set  future  standards.” 

Even  so,  this  concept  is  one  of  the  components  of  Range  Safety  Policy  for  both  the  Air 
Forces  East  Coast  and  West  Coast  test  ranges  as  described  in  their  Range  Safety  Manuals  (EWR 
127-1,  Range  Safety  Requirement,  31  Oct  1997,  p.  1-11). 


15 


RANGE  SAFETY  CRITERIA  FOR  UNMANNED  AIR  VEHICLES 
RA  TIONALE  AND  METHODOLOGY  SUPPLEMENT 

2.2.1.3  Casualty  Expectation  from  Manned  Aircraft. 

The  history  of  risk  exposure  to  people  on  the  ground  from  overflight  by  manned  aircraft  is 
measurable  in  terms  of  casualty  expectation.  Several  sources  of  mishap  rate  information  show 
that  using  1  mishap  per  million  flight  hours  is  a  reasonable  number  when  compared  to  mishap 
trends. 

Figure  2.2-1  shows  yearly  ground  fatalities  per  million  flight  hours  for  naval  aircraft 
crashes  from  1980  to  1998.  None  of  these  fatalities  were  onboard  the  mishap  aircraft.  Some  of 
the  fatalities  were  military  personnel  working  near  aircraft  operations  (such  as  the  1981  carrier 
deck  mishap),  but  others  were  not  (such  as  the  1998  Italian  cable  car  mishap).  For  the  18  years 
represented,  the  data  shows  a  mean  fatality  rate  of  1 .8  fatalities  per  million  flight  hours  due  to 
aircraft  flying  overhead. 


44  Risk  of  Aircraft  Flying  Overhead  ” 

Ground  Casualties  per  Million  Flight  Hours 
US  Navy  1980  -  1998 


Figure  2.2-1.  Ground  fatalities  for  years  1980  -1998. 


Figure  2.2-2  compares  ground  fatalities  from  Navy,  commercial,  and  general  aviation  mishaps 
per  million  flight  hours  from  1980  to  1998.  The  Navy  data  is  identical  to  the  data  shown  in 
figure  2.2-1 .  The  commercial  and  general  aviation  data  is  from  the  National  Transportation 
Safety  Board  web  site.  The  vertical  axis  is  the  mishap  rate  per  million  flight  hours  on  a 
logarithmic  scale.  The  probability  boundaries  for  “occasional,”  “remote,”  and  “improbable”  (as 
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described  in  section  2. 2. 1.1)  are  shown.  The  boxes  represent  the  ground  fatality  rate,  plus  and 
minus  one  standard  deviation  from  the  mean,  for  each  category  (military  aviation,  commercial 
aviation,  and  general  aviation). 


Civil  Aviation  Ground  Fatalities  and  U.S.  Navy  (Fatalities  on  the  Ground  and  Aboard  other  Aircraft)  Data 
for  1982-1998  plotted  according  to  the  frequency  and  severity  scale  defined  in  NAVAIRINST  6100.11  AIR- 

4.0  System  Safety  Risk  Analysis  Matrix. 


1  *10-5 


One  Catastrophic 
Event  Per  One  Million 
Flight  Hours 


1  K  10^ 


1  X.10-7 


OCCASIONAL 

REMOTE 

US  NAVY  (Ground  & 
Aboard  Other  Aircraft) 
Fatalities 

IMPROBABLE 

Figure  2.2-2.  Mishap  trend  data. 

The  mishap  trend  data  shows  that  using  a  limit  of  1  ground  fatality  per  million  flight  hours  is 
reasonable,  in  that  is  roughly  consistent  with  mishap  data. 
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2.2. 1.4  Methods  of  Calculation. 

Casualty  expectation  is  based  on  UAV  reliability  predictions  or  mishap  history,  crash 
kinetic  energy,  vehicle  dimensions,  flight  path,  and  population  along  the  flight  path.  Appendix  D 
describes  several  approaches  to  calculating  casualty  expectation. 

2.2.1.5  RCC  321-00  Alternative. 

The  Supplement  to  RCC  Document  321-00,  Common  Risk  Criteria  for  National  Test 
Ranges:  Inert  Debris ,  provides  a  detailed  approach  to  calculating  casualty  expectation.  This 
approach  is  primarily  intended  for  ballistic  missile  launches,  but  can  easily  be  adapted  to  UAVs 
in  some  situations. 

2.2.1.6  Qualitative  Alternative. 

When  empirical  data  is  not  available,  this  criterion  is  met  if  the  route  is  confined  to  sparsely 
populated  areas  and  qualitative  methods  indicate  casualty  expectation  is  negligible.  Qualitative 
methods  might  include  these  approaches: 

•  UAV  has  a  lower  mishap  rate  than  another  UAV  of  the  same  size  that  was  previously 
approved  to  fly  the  same  route. 

•  Population  density  is  sparser  than  required  to  achieve  1  casualty  per  million  flight 
hours. 

•  UAV  may  be  made  of  extremely  light  material  and  unlikely  to  cause  injury. 

•  People  potentially  exposed  to  falling  debris  are  sheltered  or  briefed  on  contingency 
procedures  in  case  of  failure. 

2.2.2  Route  Selected  to  Avoid  High  Population  Density  Area.  Routes  and  altitudes  are 
selected  to  minimize  the  possibility  of  the  UAV falling  into  a  congested  area  in  the  event  of 
electronic  or  material  malfunction.  Route  avoids  densely  populated  areas,  especially  during 
phases  of  flight  with  increased  risk. 

2.2.2.1  Congested  Area  Considerations. 

The  route  should  avoid  areas  of  high  population  density  such  as  towns,  schools,  hospitals, 
stadiums  etc.,  which  would  cause  the  momentary  casualty  expectation  to  exceed  the  acceptable 
level. 


In  most  cases,  population  density  data  can  easily  be  obtained  from  census  data.  There  may 
be  areas  within  the  census  tracts  having  a  higher  population  density  (schools,  hospitals,  stadiums 


18 


RANGE  SAFETY  CRITERIA  FOR  UNMANNED  AIR  VEHICLES 
RA  TIONALE  AND  METHODOLOGY  SUPPLEMENT 


etc.)  which  are  not  reflected  in  the  average  population  density  statistic  used  in  the  casualty 
expectation  calculation.  The  resolution  size  of  the  census  tracts  may  produce  an  inaccurate 
casualty  expectation,  which  may  appear  to  be  at  an  acceptable  level.  Therefore,  consideration  of 
additional  criteria  may  be  warranted  to  avoid  these  specific  sites.  Also,  DOD  and  FAA  policy 
guidance  directs  UAV  and  aircraft  operators  to  avoid  what  they  refer  to  as  "congested  areas. 

OPNAVINST  3710.7,  General  Naval  Training  and  Operating  Procedures  Standardization 
(NATOPS),  states:  "In  planning  and  conducting  the  flight  path  to,  in,  and  from  operating  areas, 
all  activities  operating  UAV s  shall  select  and  adhere  to  those  tracks  and  altitudes  that  completely 
minimize  the  possibility  of  UAVs  falling  into  congested  areas  in  the  event  of  electronic  or 
material  malfunction.  ”  This  instruction  also  requires  that  operations  not  create  a  perception  of 
danger  by  the  public. 

This  guidance  is  also  consistent  with  FAA  standards.  FAR  Part  91.1 19,  Minimum  Safe 
Altitudes,  states:  "Except  when  necessary  for  takeoff  or  landing,  no  person  may  operate  an 
aircraft  below  the  following  altitudes:  (a)  Anywhere.  An  altitude  allowing,  if  a  power  unit  fails, 
an  emergency  landing  without  undo  hazard  to  persons  or  property  on  the  surface,  (b)  Over 
congested  areas.  Over  any  congested  area  of  a  city,  town,  or  settlement,  or  over  any  open-air 
assembly  of  persons,  an  altitude  of  1 ,000  feet  above  the  highest  obstacle  within  a  horizontal 
radius  of  2,000  feet  of  the  aircraft,  (c)  Over  other  than  congested  areas.  An  altitude  of  500  feet 
above  the  surface,  except  over  open  water  or  sparsely  populated  areas.  In  those  cases,  the  aircraft 
may  not  be  operated  closer  than  500  feet  to  any  person,  vessel,  vehicle,  or  structure." 

2.2.2.2  High  Risk  Phases  of  Flight. 

Different  phases  and  types  of  flight  test  may  have  varying  levels  of  risk.  It  may  be 
acceptable  to  conduct  a  low  risk  operation  over  a  densely  populated  area  with  a  proven  vehicle, 
but  unacceptable  over  the  same  area  with  an  unproven  vehicle  or  during  phases  of  flight  where 
there  is  an  increased  mishap  risk. 

Some  guidelines  for  which  portions  of  a  UAV  flight  should  be  considered  high  risk 
include: 


Those  flights  where  the  probability  of  a  failure  is  unknown,  such  as  initial  flights  of  a 
new  vehicle 

Portions  of  a  flight  where  the  probability  of  failure  is  known  to  be  high  enough  to 
result  in  an  “unacceptable”  or  “undesirable”  risk  as  defined  in  the  risk  assessment 
matrix  (previously  described  in  section  1 .2). 

Portions  of  a  flight  where  this  UAV  or  similar  types  of  UAVs  have  experienced  most 
of  their  failures.  Examples  include  takeoff  and  climb-out,  and  approach  and  landing 
and  functional  check  flights. 
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•  Planned  maneuvers  intended  to  explore  the  edge  of  the  vehicle’s  performance 
envelope.  Any  unusual  maneuvers  that  could  lead  to  structural  failure,  loss  of 
propulsion,  or  loss  of  controlled  flight. 

•  Continued  flight  after  failure  of  a  redundant  flight-critical  subsystem.  For  example, 
after  failure  of  a  primary  flight  system  and  controlled  flight  is  continuing  on  a  backup 
system,  the  operators  should  consider  a  contingency  plan  a  “safer”  route  back  to  base. 

2.3  Alternatives  if  Casualty  Expectation  Criteria  Is  Not  Met. 

•  Choose  route  over  less  populated  areas. 

•  Evacuate  area  where  casualty  expectation  is  unacceptable. 

•  Verify  the  probability  of  mishap. 

•  Reduce  impact  energy  (i.e.  parachute). 

•  Investigate  the  use  of  an  FTS  to  contain  vehicle  inside  less/non  populated  areas. 

•  Investigate  Return  Home  or  other  recovery  mechanism. 

•  Investigate  shelter  factor  and  time  of  day. 

•  Request  a  waiver  from  the  Range  Commander. 

•  Cancel  the  flight. 
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3.  PROPERTY  DAMAGE  CRITERIA 

The  Property  Damage  Criteria  described  in  RCC  Document  323-99  is  an  additional 
consideration  in  determining  whether  a  UAV  is  safe  to  fly  on  a  specific  range.  The  risks 
associated  with  a  UAV  were  reviewed  by  using  the  “risk  management”  criteria  and  the 
vulnerability  of  people  at  a  specific  range  or  on  a  specific  route  of  flight  to  these  risks  was 
previously  examined  with  the  “casualty  expectation  criteria.  This  section  will  look  at  the 
vulnerability  of  property. 

Casualty  expectation  criteria  will  normally  drive  “high  risk”  operations  away  from  centers 
of  high  population  and  their  associated  properties.  Some  properties,  because  of  the  nature  of 
their  function,  are  located  in  unpopulated  areas.  Examples  are  range  assets,  hazardous  materials 
storage  sites,  and  culturally  or  environmentally  sensitive  sites.  The  “property  damage”  criteria 
ensure  that  these  sites  are  given  appropriate  consideration  when  planning  potentially  hazardous 
operations. 

Three  objectives  should  always  be  accomplished  when  reviewing  potential  for  property 
damage: 

•  Determine  what  properties  on  the  range  or  near  the  route  of  flight  are  vulnerable. 

•  Determine  what  portions  of  the  UAV  flight  are  considered  high  risk. 

•  Ensure  high-risk  portions  of  the  flight  avoid  vulnerable  properties. 

3.1  Identification  of  High  Value/High  Consequence  Properties.  The  facilities  or  properties 
that  are  vulnerable  if  a  UAV  crashes  should  be  identified  in  the  safety  approval  process.  In  terms 
of  the  hazard  risk  assessment  (previously  discussed  in  section  1 .2),  damage  to  a  facility  or 
property  is  unacceptable  if  its  damage  or  destruction  could  result  in  one  or  more  of  the  following 
severe  consequences: 

•  Loss  or  degradation  of  a  major  function 

•  Significant  monetary  loss 

•  Significant  environmental  impact 

•  Significant  cultural  impeach 

Unacceptable  loss  of  a  major  function  is  a  subjective  term  that  needs  to  be  examined  on  a 
case  by  case  basis.  Examples  of  where  loss  of  function  is  the  most  significant  consequence 
might  be  damage  to  a  satellite  farm  that  is  the  only  link  to  a  national  asset  weather  satellite  or 
damage  to  weapon  storage  areas. 

Significant  monetary  loss  is  defined  in  MIL-STD-882D  for  two  levels  of  damage  in  terms 
of  cost:  catastrophic  and  critical.  “Catastrophic”  damage  is  defined  as  $1  million  or  more; 
“Critical”  damage  is  defined  as  loss  between  $200,000  and  $1  million.  MIL-STD-882D  also 
defines  catastrophic  environmental  damage  as  “irreversible  environmental  damage  which 
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violates  law  or  regulation.  “  Critical  environmental  damage  is  damage  that  is  reversible  but 
causes  a  violation  of  law  or  regulation. 

Culturally  Sensitive  Sites  are  those  properties  having  value  in  terms  of  human  experience, 
such  as  historical  sites,  religious  sites,  monuments,  etc.  A  UAV  mishap  could  effect  cultural 
damage  that  would  adversely  impact  current  and  future  UAV  operations. 

Another  consideration  related  to  property  is  recovery  of  the  vehicle.  Some  ranges  have 
conventional  munitions  impact  areas,  which  may  be  contaminated  by  unexploded  ordnance  and 
off  limits  to  personnel.  If  a  UAV  should  fail  over  such  a  site,  its  recovery  would  be  difficult  or 
impossible. 

Ranges  that  routinely  conduct  UAV  operations  provided  examples  of  vulnerable  properties 
that  they  avoid  when  conducting  some  UAV  operations.  This  list  is  neither  exhaustive  nor  all- 
inclusive. 


TABLE  3.1-1.  VULNERABLE  PROPERTY  AND  DAMAGE  SEVERITY  RESULTS 


Vulnerable  Property 

Damage  Severity  Result 

Munitions  Testing  or  Storage  Site 

•  Catastrophic  damage  to  facility  or 
critical  monetary  loss. 

•  Loss  or  degradation  of  a  major 
function. 

NOAA  Satellite  Antenna  Farm 

•  Loss  or  degradation  of  a  major 
function. 

•  Catastrophic  or  critical  monetary  loss. 

Public  Park,  Monument  or  Property 

•  Significant  cultural  impact. 

•  Significant  environmental  impact. 

Toxic  waste  storage  site 

•  Significant  environmental  impact 

Fuel  tank  farm 

•  Initiation  of  catastrophic  or  critical 
monetary  loss 

Geothermal  power  plant 

•  Catastrophic  or  critical  monetary  loss. 

Native  American  Sites/Property 

•  Violation  of  negotiated  local  operating 
agreement,  adverse  impact  on  ability  to 
conduct  future  operations 

•  Significant  cultural  impact. 
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3.2  UAV  Route  Considerations. 


The  portions  of  the  flight  that  are  considered  “high  risk”  should  be  identified  prior  to  route 
selection  so  vulnerable  properties  can  be  avoided  during  that  portion  of  flight.  Guidelines  for 
determining  which  portions  of  the  flight  should  be  considered  “high  risk”  are  provided  in  section 
22.2.2. 

3.3  Alternatives  If  Property  Damage  Criteria  Is  Not  Met. 

•  Change  the  route  or  area  of  operation  to  avoid  the  high  consequence  property  or 
facility. 

•  Reduce  impact  energy  so  no  damage  occurs  (i.e.,  deploy  a  parachute). 

•  Remove  or  shelter  the  vulnerable  facility  if  possible. 

•  Require  use  of  an  FTS  to  ensure  vehicle  doesn’t  get  near  vulnerable  sites. 

•  Request  a  waiver  from  Range  Commander  to  accept  increased  risk. 

•  Cancel  the  flight. 
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4.  MIDAIR  COLLISION  AVOIDANCE  CRITERIA 

The  Midair  Collision  Avoidance  Criteria  described  in  RCC  Document  323-99  is  an 
additional  consideration  in  determining  whether  a  UAV  is  safe  to  fly  on  a  specific  range.  The 
risks  associated  with  a  UAV  were  reviewed  by  using  the  “risk  management”  criteria.  Previously, 
the  vulnerability  of  people  and  property  at  a  specific  range  or  on  a  specific  route  of  flight  to  these 
risks  was  examined  using  the  “casualty  expectation”  and  “property  damage”  criteria.  In  this 
section  the  vulnerability  of  other  aircraft  will  be  discussed. 

Collision  is  avoided  by  isolating  the  UAV  from  other  aircraft  or  compensating  for  see-and- 
avoid  capability  differences  with  manned  aircraft  that  increase  risk  of  collision.  The 
consequences  of  a  midair  collision  with  a  manned  aircraft  are  significant  (high  probability  of 
fatalities  and  high  cost  property  damage).  Although  flight  rules  have  evolved  for  manned  aircraft 
to  avoid  collision,  UAVs  may  or  may  not  be  compatible  with  those  rules  due  to  latency, 
visibility,  and  direct  control  issues.  Midair  collision  avoidance  criteria  focuses  attention  on  an 
examination  of  these  issues. 

4.1  Midair  Collision  Avoidance  Criteria  Case  1:  Exclusive  Use  within  Restricted  Airspace 
or  Warning  Area. 

This  criteria  is  met  if  the  UAV  is  contained  inside  restricted  airspace  or  a  warning  area, 
non-participants  are  excluded,  and  participants  are  adequately  briefed.  Such  precautions  are 
warranted  because  some  UAVs  may  not  be  able  to  see  and  avoid  other  aircraft,  or  that  ability 
may  be  unproven  in  initial  flights  of  new  vehicles.  Isolating  an  unpredictable  or  unproven 
vehicle  from  other  aircraft  ensures  there  is  no  opportunity  for  collision. 

4.1.1  UAV  Containment.  Assurance  that  the  UAV  can  be  contained  within  the  restricted  or 
warning  area  boundaries. 

Rationale:  The  UAV  must  remain  within  its  assigned  restricted  airspace  or  warning  area  so 
there  is  no  conflict  with  non-participant  aircraft  in  other  airspace. 

The  hazard  analysis  or  flight  history  of  the  UAV  may  indicate  if  there  are  failure  modes 
that  may  result  in  the  UAV  leaving  the  restricted  or  warning  area.  Consider  the  following  failure 
modes: 

•  Loss  of  navigation  information:  The  vehicle  may  have  limited  navigation  capability, 
vulnerability  to  a  single  point  navigation  system  failure,  or  the  operator  station  may 
be  limited  in  the  ability  to  recognize  a  navigation  system  discrepancy.  Operation  in  a 
backup  navigation  mode  (dead  reckoning  vs.  GPS  driven,  for  example)  may  lead  to 
significant  unrecognized  position  errors. 
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•  An  inability  to  set  local  altimeter,  unrecognized  altimeter  discrepancy,  or  inadequate 
operator  alert  for  an  altitude  deviation  may  cause  the  vehicle  to  leave  the  assigned 
altitude  limits  within  the  restricted  or  warning  area. 

•  An  inadequate  mission  planning  system  or  erroneous  mission  plan  may  lead  to  flight 
outside  of  established  boundaries. 

•  Loss  of  lift  or  loss  of  thrust  can  result  in  the  vehicle  descending  below  the  assigned 
altitude  or  the  lower  altitude  boundary  of  the  restricted  area.  Non-participant  aircraft 
below  the  restricted  area  boundary  may  be  vulnerable. 

•  Loss  of  link:  Without  direct  operator  control,  the  UAV  may  fly  outside  the  restricted 
airspace.  Emergency  mission  or  "fly  home"  routines  should  be  examined  to  ensure 
the  vehicle  will  be  contained  within  the  assigned  area  and  altitudes. 

•  Autopilot  failure  or  electrical  power  failure:  Will  the  UAV  quickly  lose  control  and 
crash  or  continue  flying  until  fuel  is  consumed? 

Review  of  the  system  maturity  of  the  vehicle,  failure  modes  possible,  and  history  of  failures 
can  help  to  determine  if  an  independent  flight  termination  system  is  required  to  keep  the  vehicle 
inside  assigned  airspace.  The  consideration  of  vehicle  operating  limits,  local  airspace  geometry, 
and  the  presence  or  absence  of  emergency  backup  systems  also  help  determine  if  an  independent 
range  flight  termination  system  must  be  mandated  to  contain  the  vehicle  within  assigned 
airspace. 

The  safety  analyst  should  verify  that  Air  Traffic  Control  (ATC)  or  the  local  military  radar 
unit  (MRU)  can  monitor  vehicle  position  for  containment  and  communicate  with  UAV 
controllers  in  a  timely  manner.  Some  portions  of  the  restricted  area  or  warning  area  may  not  be 
visible  to  air  traffic  controllers  because  of  radio  frequency  horizon  effects,  geographic 
shadowing,  or  other  limitations  of  the  monitoring  system.  The  analyst  should  ensure  the  flight  is 
restricted  to  locations  that  can  be  monitored.  The  UAV  ground  control  station  may  be  beyond 
the  communications  line  of  sight  of  the  responsible  air  traffic  control  (ATC)  or  military  radar 
unit  (MRU).  The  safety  analyst  should  ensure  both  the  primary  and  backup  communications 
links  with  ATC  are  effective. 

4.1.2  Exclusion  of  Other  Aircraft.  Assurance  that  other  aircraft  can  be  kept  out  of  the 
airspace  dedicated  to  UA  V  mission  use. 

Rationale:  To  reduce  risk,  non-participants  are  excluded  from  the  hazardous  airspace  by 
defining  hazardous  airspace  boundaries  and  activating  the  restricted  or  warning  airspace. 
Examples  of  some  approaches  currently  used  include: 

•  Declaring  predefined  portions  of  restricted  or  warning  airspace  temporarily  “exclusive 
use”  for  specific  altitudes  for  UAV  operation. 
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•  Declaring  predefined  portions  of  restricted  or  warning  airspace  temporarily  “exclusive 
use”  for  flight  of  multiple  aircraft  including  integrated  UAV  operation.  The  Flight 
Leader  is  responsible  for  aircraft  separation  within  this  airspace.  An  example  of  this 
approach  is  the  MARSA  (Military  assumes  responsibility  for  separation  of  aircraft) 
approach  used  at  Nellis  AFB. 

•  Defining  “UAV  work  areas”  in  local  procedures  manuals  and  activating  them  as 
needed. 

•  Defining  “UAV  transit  corridors”  in  local  procedures  manuals  and  activating  them  as 
needed. 


At  most  ranges,  ATC  or  MRU  should  be  able  to  monitor  the  airspace  within  and  near  the 
restricted  or  warning  area  and  communicate  (directly  or  through  controlling  agency)  with  air 
traffic  that  may  conflict.  Where  ATC  or  MRU  monitoring  capabilities  are  limited  or  do  not 
exist,  such  as  UAV  work  areas  at  remote  desert  ranges,  airspace  might  be  controlled  through 
scheduling  or  standardized  local  procedures.  Some  examples  include: 

•  The  restricted  airspace  is  remote  and,  historically,  there  has  been  no  uncontrolled 
VFR  traffic  present. 

•  The  area  to  be  flown  in  can  not  be  monitored,  but  all  approaches  to  the  area  can  be 
monitored. 

•  Visual  observation  of  the  remote  area  by  ground  observers  in  contact  with  the  UAV 
ground  control  station  can  be  used  for  low  level  operations. 

The  decision-maker  must  be  informed  of  potential  risk  associated  with  limitations  of  the 
ability  to  monitor  and  communicate  with  traffic  in  the  restricted  or  warning  areas. 

4.1.3  Participant  Coordination.  UA  V  operators  ensure  that  flight  crews  and  ATC  (or  MRU 
controllers)  understand  the  operation  as  well  as  recognize  the  limitations  of  the  UA  V.  A  local 
"standard  operating  procedure  "  may  address  routine  operations. 

Flight  crews  and  ATC  may  not  recognize  hazards  associated  with  a  UAV.  The  vehicle  may 
make  unplanned,  unusual,  or  erratic  maneuvers  due  to  normal  UAV  operation  or  control  failures, 
loss  of  link,  or  system  failure.  These  maneuvers  may  present  an  increased  risk  of  collision  with 
such  participating  aircraft  as  the  "chase"  aircraft.  Also,  the  small  size  or  stealthy  design  may 
make  it  difficult  for  participant  aircraft  to  see  the  UAV. 

A  local  SOP  that  addresses  operational  or  RDT&E  vehicles  may  be  adequate  to  ensure 
flight  crews  and  ATC  are  prepared  to  accommodate  unusual  maneuvers  or  low  visibility.  If  no 
local  SOP  applies  or  a  new  vehicle  is  significantly  different  from  UAVs  normal  for  the  area,  a 
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specific  brief  of  the  aircrew  and  /  or  ATC  brief  may  be  required  to  prepare  them  to  compensate 
for  unusual  maneuvers.  In  those  cases  where  a  UAV  is  integrated  into  a  flight  of  multiple 
participating  aircraft  and  the  Flight  Leader  is  responsible  for  separation  of  aircraft,  the  Flight 
Leader  should  ensure  flight  crews  and  ATC  are  adequately  briefed. 

4.2  Midair  Collision  Avoidance  Criteria  Case  2:  Shared  Use  within  Restricted  Airspace  or 
Warning  Areas.  The  UAV  will  be  flown  in  restricted  or  warning  areas  along  with  other  aircraft 
that  may  not  be  participating  in  the  UA  Vs  mission  or  test  event. 

This  criteria  is  met  if  the  UAV  is  contained  inside  restricted  airspace  or  a  warning  area,  and 
differences  between  UAVs  and  manned  aircraft  that  increase  risk  to  other  aircraft  (e.g.,  see-and- 
avoid  capability  deficiencies,  response  delays,  etc.)  are  accounted  for.  No  additional  FAA 
approval  is  required  for  restricted  or  warning  area  operations  conducted  in  accordance  with  FAA 
Order  7610.4. 

4.2.1  UAV  Containment.  Assurance  that  UAV  can  be  contained  within  the  restricted  or 
warning  area  boundaries. 

The  considerations  and  rationale  here  are  identical  to  what  has  previously  been  described  in 
section  4.1.1.  The  difference  here  is  that  the  airspace  control  authority  for  aircraft  within  the 
restricted  airspace  or  warning  area  will  be  different  than  outside.  The  restricted  or  warning  area 
ATC  or  MRU  will  have  limited  ability  to  direct  and  control  non-participant  aircraft  outside  the 
restricted  or  warning  area  if  a  UAV  wanders  outside  assigned  airspace. 

4.2.2  Compensating  For  See  and  Avoid  Limitations.  The  see-and-avoid  limitations  of  the 
UA  V  are  recognized  and  compensated  for.  For  example,  onboard  cameras  may  have  limitations 
(field  of  view,  sensitivity)  and  the  size  of  the  UA  V  may  make  it  difficult  for  other  aircraft  to  see. 

Rationale:  The  pilot  in  a  manned  aircraft  has  the  ability  to  look  out  for  other  aircraft  in  the 
vicinity,  but  the  UAV  pilot  may  have  limited  or  no  capability  to  see  other  aircraft.  Use  of  a 
“chase”  aircraft  as  the  UAV’s  eyes  may  improve  the  capability  of  the  UAV  to  see  other  aircraft, 
but  the  UAV  may  be  limited  in  its  ability  to  avoid  other  aircraft  because  of  time  delays  in 
controlling  the  UAV.  Even  if  the  UAV  has  a  camera,  the  instantaneous  field  of  view  may  not  be 
adequate  peripherally  to  ensure  the  complete  visual  scan  coverage  necessary  to  see-and-avoid. 

The  UAV  may  be  difficult  for  pilots  in  other  aircraft  to  see,  may  be  small  or  stealthy  in 
design,  have  a  low  visibility  paint  scheme,  or  lack  anti-collision  lights.  If  such  a  vehicle  will  be 
flying  in  a  see-and-avoid  environment  within  the  restricted  area  rather  than  “exclusive  use,”  the 
safety  analyst  should  review  the  vehicle’s  ability  to  perform  the  following  “see-and-avoid” 
functions: 

•  Traffic  detection 

•  Threat  recognition 

•  Collision  avoidance  decisions 
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•  Collision  avoidance  maneuvers 

4.2.2.1  Traffic  Detection. 


In  a  manned  aircraft,  the  pilot’s  primary  means  of  detecting  other  airborne  objects  (in  visual 
meteorological  conditions)  is  visual.  Traffic  advisory  cues  are  typically  available  from  air  traffic 
control  or  from  onboard  devices  such  as  the  Traffic  Alert  and  Collision  Avoidance  System 
(TCAS). 

In  a  UAV,  initial  detection  of  potential  traffic  might  come  from  a  number  of  sources,  which 
may  or  may  not  be  adequate.  For  example: 

•  The  chase  aircraft  has  the  same  visual  detection  ability  as  a  manned  aircraft  but  has 
the  additional  burden  of  staying  close  to  the  UAV  which  may  or  may  not  be  easy  to 
track  visually. 

•  If  a  camera  is  on  board  the  UAV  does  it  have  the  ability  to  detect  vehicles  coming 
from  several  directions  at  once,  analogous  to  a  pilots  peripheral  vision?  Does  it  have 
an  adequate  field  of  view  and  scan  rate  to  continuously  monitor  those  sectors  of  the 
vehicles  flight  path  to  adequately  detect  potential  hazards? 

•  TCAS  information  can  provide  situation  awareness  information  to  the  UAV  pilots 
ground  control  station  so  the  pilot  has  a  notion  of  what  aircraft  are  in  the  area  and  can 
anticipate  potential  collision  avoidance  maneuvers.  Is  the  vehicle  and  ground  station 
so  equipped?  Similarly,  IFF  data  repeated  to  the  pilot’s  Ground  Control  Station  from 
ATC  radar  or  airborne  platforms  such  as  AWACS  or  an  E-2  can  provide  situation 
awareness  information. 

•  A  UAV  completely  dependent  on  air  traffic  control  advisories  for  detection  of 
conflicting  traffic  does  not  constitute  the  ability  to  see-and-avoid. 

4.2.2.2  Threat  Recognition. 

The  pilot  of  a  manned  aircraft  can  visually  recognize  a  potential  collision  and  perform 
evasive  maneuvers  to  avoid  that  collision.  The  threat  is  recognized  if  the  detected  object’s 
relative  bearing  to  the  pilot’s  aircraft  does  not  change,  and  the  object  is  getting  larger.  Potential 
collision  threat  alerts  are  also  available  from  ATC  and  such  onboard  systems  as  TCAS.  A  UAV 
may  not  have  these  same  abilities.  The  safety  analyst  should  review  the  collision  threat 
recognition  capabilities  of  the  UAV  and  determine  if  they  are  adequate  for  the  situation.  Several 
considerations  for  threat  recognition  follow: 

•  Will  the  operator  use  video  camera  inputs?  Does  camera  acquisition  depend  on 
external  cueing  from  other  detection  sources?  Given  that  the  camera  sees  another 
aircraft,  does  it  have  a  demonstrated  ability  to  determine  if  the  vehicle  is  on  a 
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collision  course  or  not?  Is  it  easy  to  determine  where  the  camera  is  pointed  relative  to 
the  vehicle? 

•  Will  the  UAV  depend  on  TCAS  for  traffic  alerts?  Will  all  other  vehicles  in  the 
restricted  airspace  be  equipped  with  TCAS? 

•  A  UAV  completely  dependent  on  air  traffic  control  advisories  for  recognition  of  a 
potential  collision  does  not  constitute  the  ability  to  see-and-avoid. 

4.2.2.3  Collision  Avoidance  Decisions. 

In  a  manned  aircraft,  the  pilot  can  quickly  decide  how  best  to  avoid  a  collision  with  a 
recognized  airborne  threat  by  climbing,  diving,  changing  speed,  or  changing  heading.  In  a  UAV, 
because  of  differing  situation  awareness  implementations  and  pilot/vehicle  interfaces,  there  may 
be  delays  in  deciding  how  best  to  avoid  a  collision  and  what  action  to  take.  For  instance,  the 
operator’s  ability  to  affect  the  vehicle  may  be  limited  to  adjusting  and  uploading  a  new  flight 
plan  to  the  UAV. 

4.2. 2. 4  Collision  Avoidance  Maneuvers. 

There  may  be  a  significant  delay  in  the  ability  to  implement  a  collision  avoidance  plan  once 
the  operator  decides  what  to  do.  In  a  manned  aircraft,  the  pilot  can  quickly  and  easily  manipulate 
the  flight  controls.  In  contrast,  the  UAV  operator  may  or  may  not  have  immediate  access  to  the 
flight  controls  affecting  speed,  heading,  and  climb  or  descent.  The  operator  may  only  be  able  to 
upload  a  new  flight  plan  or  execute  a  few  canned  avoidance  maneuvers. 

Vehicles  such  as  Predator  with  a  pilot-in-the-loop  will  be  easier  to  make  quick  course, 
speed,  or  altitude  changes  to  get  out  of  the  way  than  will  vehicles  that  don't  have  a  pilot  directly 
flying  or  are  primarily  autonomous.  Also,  some  vehicles  may  be  extremely  slow  and 
cumbersome  and  relatively  less  able  to  make  nimble  collision  avoidance  maneuvers.  In  such 
cases,  the  safety  analyst  needs  to  determine  if  there  will  be  significant  delays  in  moving  the 
aircraft  and  ensure  adequate  precautions  are  made. 

4.2.2.5  Collision  Avoidance  Time  Delays. 

Obviously,  a  UAV  operator  must  be  able  to  recognize  a  potential  collision  and  maneuver 
out  of  the  way  before  the  other  aircraft  arrives.  The  relative  potential  closing  speeds  for  a  given 
type  of  airspace  and  the  distance  at  which  a  potential  collision  is  recognized  determines  the 
maximum  time  the  vehicle  operator  has  to  make  the  decision  to  maneuver  out  of  the  way. 

Time  to  maneuver  out  of  the  way  varies  from  situation  to  situation.  Some  typical  situations 
result  in  20-40  seconds  of  time  between  traffic  alert  and  potential  collision.  For  instance,  some 
restricted  areas  with  advisory  services  may  give  alerts  when  aircraft  are  5  miles  apart.  For 
tactical  jets  with  a  relative  closing  speed  of  700-900  Kts,  20-25  seconds  of  warning  time  is 
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typical.  TCAS  advisories  at  3.3  miles  of  separation  provide  20  seconds  of  warning  time  to 
vehicles  with  600  kts  of  relative  closing  speed. 

According  to  FAA  Advisory  Circular  90-48C  Pilots’  Role  in  Collision  Avoidance,  the 
nominal  time  delays  in  Table  4-1  are  typical. 

TABLE  4.2.2-1.  NOMINAL  TIMES  FOR  COLLISION 
AVOIDANCE  TASKS 


Collision  avoidance  task 

seconds 

•  Pilot  sees  object 

0.1 

•  Recognize  aircraft 

1.0 

•  Become  aware  of  collision 

5.0 

•  Decision  to  turn  left  or  right 

4.0 

•  Muscular  reaction 

0.4 

•  Aircraft  lag  time 

2.0 

Total 

12.5 

The  key  thought  here  is  that  only  seconds  are  available  to  avoid  a  collision.  A  vehicle  that 
measures  its  see-and-avoid  capability  in  a  significantly  longer  time  is  not  compatible  with  a  see- 
and-avoid  environment. 

4.2.3  Compensating  For  Delays  With  ATC  Instruction.  Vehicles  with  limited  or  no  see-and- 
avoid  capability  are  dependent  on  ATC  or  military  radar  unit  (MRU)  for  safe  separation. 
Communication  and  control  delays  may  increase  in  comparison  with  those  of  manned  aircraft. 
Vehicle  response  must  match  airspace  conditions  and  requirements. 

Rationale:  Vehicles  with  limited  or  no  see-and-avoid  capability  are  dependent  on  ATC  for  safe 
separation.  Communication  and  control  delays  may  be  longer  than  those  of  manned  aircraft  may. 
These  delays  may  decrease  or  eliminate  the  ability  of  the  vehicle  to  respond  to  ATC  direction  in 
a  timely  manner.  If  vehicle  response  does  not  match  airspace  conditions  and  requirements,  there 
is  increased  risk  of  collision. 

The  design  of  the  UAV  may  include  time  delays  in  the  downlink  of  information  to  the  air 
vehicle  controller  or  in  the  uplands  of  the  controller’s  commands  to  the  vehicle.  The  time  delay 
in  the  communications  link  between  ATC  and  the  air  vehicle  operator  can  also  be  an  issue. 
Examples  of  sources  of  delays  can  include: 

•  An  unusual  ATC-to-vehicle  ground  station  link  -  The  normal  link  is  UHF  or  VHF 
radio  direct  from  aircraft  to  ATC.  The  UAV  operator  may  be  beyond  line  of  sight  of 
the  ATC  facility,  and  may  have  to  depend  on  a  telephone  or  SATCOM  relay  rather 
than  radio  direct  from  the  aircraft. 
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•  Non-deterministic  software  in  the  vehicle  ground  station  may  delay  the  display  of 
decision  information  to  the  operator,  or  may  delay  transmission  of  critical  flight 
commands. 

•  Human  interface  with  the  vehicle:  some  vehicles  may  require  the  operator  to  type  in  a 
new  waypoint  or  flight  plan  to  make  a  collision  avoidance  course  change. 

•  Distance  in  communications  link,  especially  if  the  command  links  use  satellites. 

•  UAVs  operating  “autonomously”:  There  may  not  be  an  operator  monitoring,  or  the 
vehicle  may  have  lost  its  link  to  the  ground  station. 

Each  of  these  examples  can  result  in  delays  in  recognizing  a  potential  collision  or  a  delay  in 
sending  collision  avoidance  commands  to  the  UAV . 

4.3  Midair  Collision  Avoidance  Criteria  Case  3:  UAV  Operations  In  Other  Than 
Restricted  and  Warning  Areas.  UA  V plans  to  enter  National  Airspace,  other  than  restricted 
area  or  warning  area.  FAA  is  responsible  for  aircraft  separation  and  must  authorize  and 
approve  the  flight. 

This  criteria  is  met  with  both  (1)  documentation  of  FAA  approval  and  (2)  review  and 
approval  by  the  accountable  government  sponsor. 

4.3.1  FAA  Approval.  UAVs  that  plan  to  enter  the  National  Airspace  System  shall  conform  to 
FAA  regulations  and  gain  approval  from  the  regional  FAA  representative.  A  Certificate  of 
Authorization  is  required. 

Rationale:  Flights  that  require  special  FAA  approval  are  described  in  FAA  Order  7610.4, 
Special  Military  Procedures.  In  general,  any  UAV  flights  outside  of  restricted  areas  or  warning 
areas  will  require  approval.  Users  should  coordinate  early  in  the  planning  stages  with  the  local 
FAA  representative  to  identify  the  exact  requirements. 

Note:  The  FAA  refers  to  unmanned  air  vehicles  as  "remotely  operated  aircraft"  or  ROAs  that 
must  comply  with  Federal  Aviation  Regulations  like  other  aircraft. 

The  process  (repeated  below)  for  getting  FAA  approval  in  the  form  of  a  "Certificate  of 
Authorization"  is  described  in  FAA  Order  7610.4J  Change  1,  dated  3  July  2000,  entitled 
SPECIAL  MILITARY  OPERATIONS. 

"ROAs  operating  outside  Restricted  Areas  and  Warning  Areas  shall  comply  with  the 
following: 
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а.  At  least  60  days  prior  to  the  proposed  commencement  of  ROA  operations,  the  proponent 
shall  submit  an  application  for  a  Certificate  of  Authorization  (CO A)  to  the  Air  Traffic  Division 
of  the  appropriate  FAA  regional  office.  CO  A  guidance  can  be  found  in  FAA  Handbook  7210.3, 
Facility  Operation  and  Administration,  Part  6,  Chapter  18,  Waivers,  Authorizations, 

Exemptions,  and  Flight  Restrictions.  The  following  documentation  should  be  included  in  the 
request: 

NOTE  -  In  the  event  of  real-time,  short  notice,  contingency  operations,  this  lead  time  may  be 
reduced  to  the  absolute  minimum  necessary  to  safely  accomplish  the  mission. 

1.  Detailed  description  of  the  intended flight  operation  including  the  classification  of  the 
airspace  to  be  utilized. 

2.  ROA  physical  characteristics. 

3.  Flight  performance  characteristics. 

4.  Method  of pilotage  and  proposed  method  to  avoid  other  traffic. 

5.  Coordination  procedures. 

б.  Communications  procedures. 

7.  Route  and  altitude  procedures. 

8.  Lost  link/mission  abort  procedures. 

9.  A  statement  from  the  DOD  proponent  that  the  ROA  is  ‘airworthy  ’.  “ 

4.3.2  DOD/NASA  Review.  Government  sponsor  (i.e.  the  DOD  or  NASA)  must  also  review  and 
approve  if  there  is  any  DOD  or  NASA  liability.  Differences  between  UA  Vs  and  manned  aircraft 
(e.g,  see-and-avoid,  and  response  delays)  must  be  accounted  for. 

For  RDT&E  vehicles  operating  from  MRTFB  ranges  in  accordance  with  DOD  Directive 
3200.1 1 ,  the  Range  Commander  has  overall  responsibility  for  UAV  flight  safety.  For 
operational  vehicles,  the  operational  unit  Commanding  Officer  has  ultimate  responsibility  for 
complying  with  local  range  regulations  while  on  the  range  and  FAA  regulations  when  outside  the 
range.  According  to  FAA  Order  7610.4J  Change  1  3  July  2000: 

.  “The  proponent  and/or  its  representatives  shall  be  noted  as  responsible  at  all  times  for 
collision  avoidance  maneuvers  with  nonparticipating  aircraft  and  the  safety  of  persons  or 
property  on  the  surface.  ” 

4.3.2.1  UAV  Containment.  Assurance  that  UA  V  can  be  contained  within  the  boundaries  of  the 
pre-planned  route  of  flight  defined  in  the  flight  plan  and  approved  by  the  FAA. 

Rationale:  The  considerations  and  rationale  here  are  similar  to  what  has  previously  been 
described  in  sections  4.1.1  and  4.2.1.  The  difference  here  is  the  route  may  extend  for  a  longer 
distance  from  the  ground  station,  and  local  weather  and  air  traffic  information  may  be  more 
difficult  to  obtain.  There  may  be  less  maneuvering  room  to  accommodate  a  vehicle  which  may 
be  less  predictable  than  a  manned  aircraft.  The  operator  must  maintain  the  vehicle  within  a  pre- 
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planned  route  of  flight  so  there  is  no  conflict  with  other  aircraft  or  other  Special  Use  Airspace 
(SUA). 

The  UAV  ground  control  station  may  be  beyond  the  communications  line  of  sight  of  the 
responsible  ATC  or  MRU.  Ensure  that  both  the  primary  and  backup  communications  links  are 
effective  for  the  entire  route  of  flight  and  any  pre-planned  emergency  routes. 

4.3.2.2  Compensating  For  See-and-Avoid  Limitations.  The  limitations  of  the  UAV  are 

recognized  and  compensated  for.  For  example,  onboard  cameras  may  have  limitations  (field  of 
view,  sensitivity)  and  the  size  of  the  UAV  may  make  it  difficult  for  other  aircraft  to  see. 

Rationale:  The  considerations  and  rationale  here  are  similar  to  what  has  previously  been 
described  in  sections  4.2.2.  This  is  a  key  area  of  concern  in  the  FAA  approval. process.  In  FAA 
Order  7610.4,  a  see-and-avoid  capability  with  equivalent  levels  of  safety  is  mandated  as  follows: 

"Approvals  for  ROA  operations  should  require  the  proponent  to  provide  the  ROA  with  a 
method  that  provides  an  equivalent  level  of  safety,  comparable  to  see-and-avoid  requirements  for 
manned  aircraft. 

Methods  to  consider  include,  but  are  not  limited  to  radar  observation,  forward  or  side 
looking  cameras,  electronic  detection  systems,  visual  observation  from  one  or  more  ground  sites 
monitored  by  patrol  or  chase  aircraft,  or  a  combination  thereof  " 

This  same  order  also  mandates  use  of  anticollision  lights,  strobe  lights,  and  IFF : 

"c.  ROAs  shall  be  equipped  with  standard  aircraft  position  lights  and  high  intensity  strobe 
lights  in  accordance  with  criteria  stipulated  in  14  CFR,  section  23.1401.  These  lights  shall  be 
operated  during  all  phases  of  flight  in  order  to  enhance  flight  safety. 

d.  ROAs  shall  be  equipped  with  an  altitude  encoding  transponder  that  meets  the 
specifications  of  14  CFR,  section  91.215.  The  transponder  shall  be  set  to  operate  on  a  code 
assigned  by  air  traffic  control.  Unless  the  use  of  a  specific,  special-use  code  is  authorized,  the 
ROA  pilot-in-command  shall  have  the  capability  to  reset  the  transponder  code  while  the  ROA  is 
airborne.  If  the  transponder  becomes  inoperative,  at  the  discretion  of  the  affected  region  or  air 
traffic  facility,  the  mission  may  be  canceled  and/or  recalled. " 

4.3.2.3  Compensating  For  Delays  With  ATC  Instruction.  Vehicles  with  limited  or  no  see- 
and-avoid  capability  are  dependent  on  ATC  for  safe  separation.  Communication  and  control 
delays  may  increase  in  comparison  with  those  of  manned  aircraft.  Vehicle  response  must  match 
airspace  conditions  and  requirements. 

Rationale:  The  considerations  and  rationale  here  are  identical  to  what  has  previously  been 
described  in  section  4.2.3.  The  difference  here  is  the  FAA  requires  an  "instantaneous"  response 
as  described  in  FAA  order  7610.4: 
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"e.  Instantaneous  two-way  radio  communication  with  all  affected  ATC facilities  is  required. 
For  limited  range,  short  duration  flights,  proponents  may  request  relief from  radio  requirements 
provided  a  suitable  means  of  alternate  communication  is  available.  Compliance  with  all  ATC 
clearances  is  mandatory. " 
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5.  CRITERIA  FOR  RELIABILITY  AND  ADEQUACY  OF  SAFEGUARDS 

There  must  be  evidence  to  show  that  key  safeguards  will  mitigate  critical  or  severe  risks. 
Safeguards  must  be  provided  if  the  hazard  analysis  requires  it  or  if  the  UAV  or  test  operation 
does  not  meet  other  safety  criteria  (e.g.,  casualty  expectation,  property  damage,  collision 
avoidance)  without  it.  Typical  systems  that  may  be  considered  as  safeguards  include,  but  are  not 
limited  to: 


•  Emergency  remote  pilots 

•  Flight  termination  systems 

•  Software  "fly  home"  software  routines 

•  Parachutes 

Procedures  that  are  considered  safeguards  include  emergency  procedures,  checklists  that  address 
safety  critical  systems,  and  documented  warnings  and  cautions. 

ALTERNATIVES  IF  CRITERIA  NOT  MET: 

The  following  alternatives  apply  to  hardware,  software,  and  procedural  safeguards: 

•  Restrict  operation  to  avoid  specific  hazard 

•  Add  an  alternative  safeguard  to  address  the  specific  hazard 

•  Request  a  waiver  from  Range  Commander  to  accept  increased  risk. 

•  Cancel  the  flight 
Additional  guidance  is  provided  below. 

5.1  Hardware  Safeguards.  Evidence  must  show  that  the  reliability  of  key  hardware  safeguards 
is  adequate.  The  range  may  require  one  or  more  of  the  following: 

•  Show  evidence  of  a  reliability  of  0.999  at  95%  confidence  level  in  a 
representative  environment. 

Rationale:  This  reliability  number  (0.999  at  95%  confidence)  is  the  overall  reliability  goal  for 
flight  termination  systems.  The  same  goal  can  be  used  for  other  than  FTS  systems  for  safety 
critical  applications.  According  to  the  FTS  standard  (RCC  Standard  319-99)  system  reliability  is 
demonstrated  by: 

“(1)  Designing  the  system  to  be  single  fault  tolerant 
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(2)  Performing  qualification,  acceptance,  certification,  and  pre-mission  testing  in 

accordance  with  the  FTS  standard 

(3)  Maintaining  strict  quality  control  practices  during  fabrication,  test,  installation,  and  test. 

(4)  Performing  a  reliability  prediction  to  show  0.999  probability  is  met.  Use  150%  of 

mission  time  and  analysis  in  accordance  with  MIL-HDBK-217E  Reliability 
Prediction  of  Electronic  Equipment,  using  the  applicable  environmental  factor.” 

Refer  to  RCC  Standard  319-99  Chapter  4;  “RPV,  Sub  Scale  and  RLV”;  Section  4.4.17, 
Reliability 


•  FTS  subsystems  meet  the  current  RCC  flight  termination  standard  (i.e.,  RCC 
Standard  319-99  or  equivalent) 

Rationale:  If  the  hazard  analysis  indicates  a  flight  termination  system  is  required,  a  system  that 
meets  the  RCC  Standard  3 1 9-99  requirements  should  be  acceptable  at  MRTFB  ranges. 

•  The  safeguard  subsystem  meets  an  established  reliability  standard  for  that  type  of 
safeguard.  (Define  as  an  example  the  reliability  of  a  typical  FTS,  which  is  required 
by  RCC  Standard  3 1 9-99,  or  the  FAA.) 

Rationale:  If  the  safeguard  is  not  a  flight  termination  system,  but  is  instead  something  not 
covered  by  RCC-319,  the  use  of  an  industry  standard  related  to  that  type  of  hardware  may  be 
appropriate.  If  the  industry  standard  addresses  the  environment  the  system  may  be  exposed  to, 
there  is  then  a  basis  for  making  an  informed  decision  on  system  reliability. 

•  The  system  or  safeguard  has  been  tested  and  can  be  monitored  in  flight  or  will  be 
explicitly  checked  before  flight. 

Rationale:  New  systems  that  have  no  industry  standard  can  be  used  if  the  hazards  are 
recognized  and  attention  focused  on  the  testing,  pre-flight  inspection,  and  in-flight  monitoring  of 
the  system. 

5.2  Software  Safeguards.  Evidence  must  show  that  the  reliability  of  key  software  safeguards  is 
adequate.  Examples  of  software  safeguards  may  include  “Fly  home  ”  or  "emergency  mission" 
routines  in  the  event  of  lost  link,  and  some  “ emergency  remote  pilot”  components. 

The  range  user’s  risk  management  plan,  as  described  in  section  1  of  this  document,  should 
identify  if  there  are  failure  modes  that  are  mitigated  with  software.  If  there  are  software 
functions  that  address  critical  hazards,  the  range  safety  analyst  needs  to  know  that  the  software 
function  will  work  when  required.  The  basic  questions  to  be  answered  are  as  follows: 

•  Have  all  safety  critical  requirements  been  identified?  Has  the  UAV  been  subjected  to 
a  software  safety  program?  Have  software  functions  been  addressed  in  the  hazard 
analyses? 
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•  Have  safety  critical  software  requirements  identified  in  the  software  safety  program  or 
hazard  analyses  been  implemented? 

•  What  assurance  is  there  that  these  implemented  requirements  will  work?  Have  they 
been  tested?  Can  these  safety  critical  software  functions  be  tested  before  flight  or 
monitored  in  flight? 

Detailed  guidance  on  software  safety  issues  can  be  found  in  the  Software  Safety  Handbook, 
Joint  Software  Safety  Committee,  and  in  NASA’s  Guidebook  for  Safety  Critical  Software  - 
Analysis  and  Development. 

5.3  Procedural  Safeguards.  Evidence  must  show  procedural  safeguards  are  adequate. 
Examples  of  procedural  safeguards  are  emergency  procedures,  checklists,  operator  certification, 
and  training. 

•  Operator  procedures  that  will  be  used  as  a  safeguard  must  be  documented. 

•  Procedures  must  have  been  reviewed  and  approved  by  the  Range  Commander  or 
delegated  representative. 

Rationale:  When  a  malfunction  occurs,  if  the  operator  can  respond  quickly  and  accurately,  the 
probability  increases  that  the  vehicle  can  be  recovered  safely  or  that  damage  can  be  minimized. 
The  implications  of  specific  safety  critical  failures  are  best  considered  beforehand,  when  system 
experts  can  lay  out  the  best  choices  for  the  operators.  Written  procedures  also  allow  the  range  to 
verify  that  procedures  are  compatible  with  local  conditions.  Checklists  for  specific  safety  critical 
procedures  help  to  ensure  complicated  actions  are  performed  correctly.  Training  and  operator 
certification  helps  to  ensure  safety  critical  procedures  are  properly  accomplished  when  required. 
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REFERENCES  AND  INFORMATION  SOURCES 
A.l  RISK  MANAGEMENT  REFERENCES  AND  INFORMATION  SOURCES 


AFI  91-213,  Operational  Risk  Management  Program 

DOD  DIRECTIVE  3200.1 1 ,  Major  Range  and  Test  Facility  Base 

MIL-STD-882,  System  Safety 

NHB  1700.1  (Vl-B),  NASA  Safety  Policy  and  Requirements  Document,  1993: 

http://nodis.hg.nasa.gov/Librarv/Directives/NASA-WIDE/Procedures/contents.html 

OPNAVINST  3500.39,  Introduction  To  Operational  Risk  Management 
For  further  information  on  Risk  Management: 


Army  Safety  Center:  http://safety.army.mil/home.html 

Army  Risk  Management  Information  Center:  http://rmis.army.mil/ 


Air  Force  Safety  Center:  http://rmis.saia.af.mil/ 


Air  Force  ORM  Pubs: 

AFI  91-213,  Operational  Risk  Management  (ORM)  Program 

AFP  AM  91-214,  Operational  Risk  Management  (ORM)  Implementation  and  Execution 
AFP  AM  91-215,  Operational  Risk  Management  (ORM)  Guidelines  and  Tools: 
http://afftc.edwards.af.mil/pim/afmenu/91series.htm 


NASA  Continuous  Risk  Management: 

http://satc.gsfc.nasa.gov/support/ASM  FEB99/crm  at  nasa.html 

Navy  Safety  Center/ORM: 

http :// www.  safety  center  .navy  .mi  1/ORM/ ormmain.htm 


USMC  ORM: 

http://www.hqmc.usmc.milhttp://www.hqmc.usmc.mil/safety.nsf/852564750060e4c8852 

5645d006f6979/fd7ddc822da34c0fB52564290069ba99?QpenDocument 
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A.2  CASUALTY  EXPECTATION  REFERENCES  AND  INFORMATION  SOURCES 


Title  14  Code  of  Federal  Regulations,  Federal  Aviation  Regulations 

MIL-STD-882D,  Department  of  Defense  Standard  Practice  for  System  Safety,  1 0  February  2000 
EWR  127-1,  Range  Safety  Requirements,  31  Oct  1997, 45th  Space  Wing,  Patrick  AFB  FL 
Public  Law  81-60,  Legislative  History,  81st  Congress,  pg.  1235 

NAVAIR  Instruction  5 1 00. 1 1 ,  Research  and  Engineering  Technical  Review  of  Risk  Process  and 
Procedures  for  Processing  Grounding  Bulletins 

RCC  Standard  321-00,  Common  Risk  Criteria  for  National  Test  Ranges:  Inert  Debris 

For  further  information: 

Air  Force  Safety  Center:  http://www-afsc.saia.af.mil/ 

Navy  Safety  Center/ORM;  http://www.safetycenter.navy.mil/ 

National  Transportation  Safety  Board:  http://www.ntsb.gov/aviation 
Range  Commanders  Council:  http://ics.mil/RCC 
A.3  PROPERTY  DAMAGE  REFERENCES 

MIL-STD-882D,  Department  of  Defense  Standard  Practice  for  System  Safety,  10  February  2000 

A.4  COLLISION  AVOIDANCE  REFERENCES  AND  INFORMATION  SOURCES 

Title  14,  Code  of  Federal  Regulations,  Federal  Aviation  Regulations 

FAA  Order  71 10.65M  Change  1,10  August  2000,  Air  Traffic  Control 

FAA  Order  7610.4J  Change  1,  3  July  2000,  Special  Military  Operations 

FAA  Advisory  Circular  AC  90-48C,  Pilot’s  Role  in  Collision  Avoidance 
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For  Further  Information: 


FAA  Home  Page:  http://www.faa.gov 

FAA  Publications  Library:  http://www.faa.gov/atpubs/default.htm 

Federal  Aviation  Regulations:  http://www.faa.gov/avr/AFS/FARS/far  idx.htm 

TCAS  Information: 

FAA  TCAS  and  ADSB  Web  Page:  http://adsb.tc.faa.gov/ 

MITRE  Inc:  http://www.mitre.org/pubs/showcase/tcas/tcas.html 

A.5  SAFEGUARDS  REFERENCES  AND  INFORMATION  SOURCES 

NAS A-STD-87 1 9. 1 3 A,  NASA  Software  Safety  Standard: 
http://satc.gsfc.nasa.gov/assure/nss87 1 9  1 3  .html 

NASA-GB- 1740. 13-96,  ASA  Guidebook  for  Safety  Critical  Software  Analysis  and 
Development:  http://www.iw.nasa.gov/SWG/resources/SWG  safety.html 

STAN  AG  4044,  NATO  Standardization  Agreement,  Safety  Design  Requirements  and  Guidelines 
for  Munitions  Related  Safety  Critical  Computing  Systems 

Software  Safety  Handbook,  Joint  Software  System  Safety  Committee,  December  1999: 
http  ://www.nswc  .navy  .mil/safety 

IEC  1 508,  Functional  Safety,  Safety-Related  Systems,  International  Electrotechnical  Committee 
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B.l  INTRODUCTION  TO  REVIEW  QUESTIONS .  B-3 

B.2.  UAV  BACKGROUND  INFORMATION  .  B-4 

B.2.1  Vehicle  Description .  B-4 

B.2.2  Vehicle  Performance . B-4 

B.2.3  Vehicle  Safety  History  and  Reliability . B-4 

B.2.4  Operator  Qualifications .  B-5 

B.2.5  Hazardous  Materials .  B-5 


B.3.  CAUSES  OF  “LOSS  OF  CONTROL”  .  B-5 

B.3.1.  Loss  of  Command  Uplink  .  B-5 

B.3.2.  Loss  of  Vehicle  Position  Information  .  B-6 

B.3.3.  Loss  of  Flight  Reference  Data .  B-7 

B.3.4.  Unresponsive  Flight  Controls  .  B-7 

B.3.5.  Loss  of  Propulsion .  B-7 

B.3.6.  Loss  of  Electrical  Power  .  B-8 

B.3.7.  Ground  Control  Station  Failures . .  B-8 


B.4.  REVIEW  Of  COMMON  SAFEGUARDS  .  B-8 

B.4.1  Degraded  Modes  of  Flight .  B-9 

B.4.2  Return  Home  Modes  .  B-9 

B.4.3  Ditching . B-10 

B.4.4  Flight  Termination  System . B-10 

B.4.5  Fail  Safe . B-12 

B.4.6  Parachute . B-12 


B.5.  QUESTIONS  ABOUT  “MIDAIR  COLLISION”  HAZARDS  . B-12 

B.5.1  Exclusive  Airspace . B-12 

B.5.2  UAV  Routes  . -  B-13 

B.5.3  Collision  Avoidance  System . B-13 

B.5.4  Interaction  with  Air  Traffic  Control . B-13 


B.l  INTRODUCTION  TO  REVIEW  QUESTIONS 


Range  Safety  is  tasked  to  identify  potential  hazards  on  the  range  and  ensure  safeguards  are 
put  in  place  to  reduce  risk  to  an  acceptable  level,  consistent  with  existing  local  policy  guidance. 

If  the  operational  risks  of  a  specific  program  exceed  specified  levels  even  after  implementation 
of  reasonable  safeguards,  a  waiver  decision  is  required  from  the  local  Range  Commander. 

This  is  a  "living  document"  intended  as  a  tool  for  Range  Safety  to  evaluate  new  and 
ongoing  UAV  test  programs.  The  document  will  help  ensure  the  local  range  commander  is  fully 
advised  and  informed  of  all  known  risks.  It  also  serves  as  a  consistent  approach  to  UAV 
program  range  safety  reviews. 

This  appendix  is  focused  on  hazards  that  may  result  in  the  following  consequences: 

•  UAV  crashes  which  may  result  in  death  or  injury,  or  damage  to  property. 

•  Mid-air  collision  between  UAV  and  manned  aircraft  causing  death  or  injury  to  pilot, 
or  damage  to  manned  aircraft. 

Each  section  provides  questions,  based  on  past  experience  and  lessons  learned  from  other 
programs,  which  focus  on  hazards  and  safeguards  as  outlined  below: 

Section  B.2:  UAV  background  information 

Section  B.3:  Potential  causes  of  vehicle  loss  of  control  that  may  result  in  a  crash  or  flight 
into  non-exclusive  airspace. 

Section  B.4:  Common  safeguards  and  emergency  procedures  to  prevent  an  uncontrolled 
crash  off  range  or  mid-air  collision. 

Section  B.5:  The  midair  collision  hazard  and  system  interaction  with  Air  Traffic  Control. 
Successful  completion  of  this  review  process  will  result  in  confidence  that: 

•  Key  system  vulnerabilities  have  been  identified 

•  Safeguards  have  been  verified  to  exist  for  these  system  vulnerabilities 

•  Safeguards  are  adequate,  and 

•  Deficiencies  or  inadequacies  of  the  proposed  safeguards  have  been  recognized 

When  the  review  is  completed,  the  safety  analyst  will  have  enough  information  to  clearly  tell  the 
project  what  deficiencies  they  must  fix,  to  document  for  the  Range  Commander  the  areas  of  risk, 
and  to  recognize  the  key  range  safety  issues  to  monitor  during  the  test. 
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B.2  UAV  BACKGROUND  INFORMATION 


Background  information  about  the  UAV  system  is  required  to  understand  the  system  well 
enough  to  make  a  defensible  risk  assessment.  This  background  information  is  used  as  a  starting 
point  for  identifying  potential  system  hazards  and  reviewing  existing  system  safeguards.  Items 
listed  below  are  basic  guidelines  with  potential  reference  sources  that  are  helpful  in  satisfying  the 
requirement  for  understanding  the  system. 

B.2.1  Vehicle  Description. 

•  Users  handbook  (NATOPS  equivalent) 

•  Physical  dimensions 

•  Weight  (empty  and  max) 

•  Mission  description 

•  Crew  requirements 

•  Description  of  command  and  control  system 

•  List  of  hazardous  material  associated  with  this  vehicle 

B.2.2  Vehicle  Performance. 


•  Performance  charts 

•  Max  altitude 

•  Max  endurance 

•  Max  range 

•  Range  vs.  altitude  (glide) 

•  Cruise  speed 

•  Max  speed 

•  Rate  of  climb,  rate  of  descent 

B.2.3  Vehicle  Safety  History  and  Reliability. 

Mishap  history:  What  is  the  flight  history  of  this  model  UAV?  How  many  crashes  and 
failures  have  occurred?  What  has  been  the  corrective  action  to  ensure  the  failures  do  not  occur 
again? 

Any  hazard  analyses  from  contractor  or  system  safety? 

Is  there  an  estimate  for  system  mean  time  between  failure?  How  has  this  MTBF  been 
determined  (analysis  or  actual  data)? 

What  performance  or  environmental  limitations  were  used  to  estimate  system  MTBF?  Will  the 
proposed  test  exceed  any  of  these  limitations? 
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Is  there  a  software  safety  program  for  this  UAV  system?  What  flight  critical  components  are 
software  controlled?  Have  software  safety  analyses  been  performed? 

B.2.4  Operator  Qualifications. 

What  personnel  are  involved  in  the  mission  and  what  are  their  functions?  What  information  do 
they  have  to  make  safety-related  decisions? 

What  is  the  basis  of  the  qualification  of  the  vehicle  operators?  How  much  experience  do  they 
have?  How  recently  have  they  flown  this  type  vehicle? 

B.2.5  Hazardous  Materials. 

Any  hazardous  materials  onboard  (flammable,  toxic,  energy  storage,  ordnance)? 

Can  a  crash  start  a  fire? 

B.3.  CAUSES  OF  “LOSS  OF  CONTROL” 

Vehicle  loss  of  control  can  easily  result  in  a  mishap.  If  we  can  identify  any  potential  causes  of 
"loss  of  control"  that  may  have  been  overlooked,  safeguards  can  be  applied,  or  test  conditions 
can  be  restricted  to  reduce  risk  to  an  acceptable  level. 

The  following  questions  focus  on  system  vulnerabilities  previously  experienced,  some  of  which 
have  resulted  in  mishaps. 

B.3.1  Loss  of  Command  Links. 

What  happens  when  command  link  is  lost? 

How  does  vehicle  respond  if  link  is  never  re-established? 

How  does  the  vehicle  recognize  that  loss  of  command  link  has  occurred? 

How  does  the  UAV  operator  in  the  ground  control  station  recognize  loss  of  command  link  has 
occurred? 

B.3.1. 1  Backup  Communications  Links. 

Is  there  a  backup  command  transmitter  and  receiver? 

Does  the  backup  transmitter  have  the  same  or  more  “effective  radiated  power”? 


B.3.1.2  Link  Analysis. 


Has  RF  link  analysis  been  performed  to  verify  both  primary  and  backup  transmitters  can 
communicate  with  the  vehicle  at  the  furthest  point  in  its  planned  operation? 

Does  link  analysis  address  all  RF  links? 

•  Uplinks  from  primary  and  backup  ground  stations 

•  Secondary  uplinks  from  each  ground  station 

•  Downlinks  to  primary  and  backup  ground  stations 

•  Flight  Termination  Link 

Does  link  analysis  consider  RF  horizon? 

Is  maximum  range  for  each  link  explicitly  stated? 

Is  there  at  least  12  dB  of  signal  excess  in  FTS  link? 

How  do  you  determine  if  the  primary  and  backup  transmitters  are  radiating  specified  output 
power? 

How  do  you  determine  if  the  vehicle  primary  and  backup  command  and  control  receivers  and 
FTS  receivers  are  operating  at  specified  sensitivity? 

Are  there  any  nulls  in  the  command  transmitter  antenna  pattern?  Do  the  operators  know  where 
they  are? 

Are  there  areas  of  RF  masking  due  to  location  of  antennas  on  the  UAV  relative  to  their  position 
and  to  ground  station  antennas?  Are  there  RF  null  spots  based  on  orientation  of  the  UAV? 

What  is  the  link  susceptibility  to  multipath?  What  is  the  system  response  if  multipath  is 
experienced? 

B.3.1.3  Radio  Frequency  Interference  (RFI). 

What  is  the  effect  of  RFI  on  the  command  and  control  system? 

Is  there  a  frequency  allocation  for  all  RF  links? 

What  frequency  does  the  UAV  system  operate  on  and  does  this  cause  any  interference  with  any 
other  local  systems? 

Is  the  backup  command  link  sufficiently  protected  from  spurious  command  signals? 

B.3.2  Loss  of  Vehicle  Position  Information. 
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What  are  the  sources  of  vehicle  navigation  position  information  to  the  UAV  operator?  Are  there 
redundant  sources  so  the  UAV  operator  can  tell  if  there  is  a  discrepancy? 

If  the  UAV  operator  loses  primary  position  information,  is  control  also  lost? 

Does  the  UAV  operator  have  access  to  any  external  sources  of  position  information  that  could 
serve  as  a  backup  (radar,  IFF,  binoculars)? 

How  does  the  vehicle  autopilot  respond  to  loss  of  primary  internal  navigation  source?  Is  there  a 
backup?  What  are  the  indications  in  the  ground  station  to  the  UAV  operator? 

B.3.3  Loss  of  Flight  Reference  Data. 

What  are  the  on-board  sources  of  position,  attitude,  heading,  altitude,  and  airspeed  information  to 
the  UAV  operator  and/or  autopilot? 

How  does  the  vehicle  autopilot  respond  to  loss  of  primary  attitude  source?  Is  there  a  backup? 
What  are  the  indications  to  the  UAV  operator? 

How  does  the  vehicle  autopilot  respond  to  loss  of  primary  heading  source?  Is  there  a  backup? 
What  are  the  indications  to  the  UAV  operator? 

How  does  the  vehicle  autopilot  respond  to  loss  of  primary  altitude  source?  Is  there  a  backup? 
What  are  the  indications  to  the  UAV  operator? 

How  does  the  vehicle  autopilot  respond  to  loss  of  primary  airspeed  source?  Is  there  a  backup? 
What  are  the  indications  to  the  UAV  operator? 

B.3.4  Unresponsive  Flight  Controls. 

What  will  happen  if  a  servo  or  flight  control  sticks  or  becomes  unresponsive?  How  does  the 
autopilot  respond?  Is  there  a  backup?  How  quickly  will  the  UAV  operator  recognize  this? 

What  happens  if  the  throttle  is  stuck?  How  will  the  UAV  operator  recognize  this  condition?  Is 
there  a  recovery  procedure? 

B.3.5  Loss  of  Propulsion, 

What  happens  to  the  vehicle  when  propulsion  stops? 

Will  sufficient  velocity  and  electrical  power  remain  for  “controlled  ditch”  or  “dead  stick 
landing”? 


Can  the  engine  be  restarted  in  flight? 
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Is  the  propulsion  system  affected  by  environmental  conditions  (temperature,  icing,  dust,  etc.)? 
What  are  the  limits?  Are  the  limits  and  failure  modes  confirmed  by  test  data?  Are  limits 
considered  in  test  plan? 

How  is  fuel  volume  or  fuel  utilization  monitored? 

B.3.6  Loss  of  Electrical  Power. 

What  happens  when  primary  electrical  power  is  lost? 

Is  there  a  separate  battery  bus?  What  does  battery  bus  power?  Does  automatic  system  load 
shedding  occur  if  power  is  reduced?  Are  there  "essential  busses"  for  reduced  power  operations? 

Are  all  flight  essential  systems  on  an  essential  bus? 

Is  there  a  battery  power  available  time  limit  associated  with  loss  of  electrical  power?  How  long? 
What  if  the  UAV  is  too  far  from  base  to  get  back  before  power  runs  out? 

Does  FTS  activate  if  battery  backup  fails  (i.e.,  fails  “safe”)? 

Does  FTS  operate  on  an  independent  battery  circuit? 

How  is  backup  battery  checked  prior  to  takeoff? 

Safety  backup  system  battery  lifetime  is  a  critical  issue.  How  do  you  know  how  much 
emergency  battery  power  is  left?  Is  battery  usage  data  available  on  telemetry?  Is  a  battery  use 
log  kept? 

B.3.7  Ground  Control  Station. 

What  is  the  source  of  electrical  power  for  the  ground  control  station?  Is  there  an  un-interruptable 
backup  power  source? 

What  happens  if  electrical  power  is  lost? 

Do  backup  command  transmitter  and  emergency  systems  have  adequate  protection  from  loss  of 
electrical  power? 

If  power  to  the  ground  station  is  lost,  does  it  affect  how  flight  information  is  calculated?  Do  all 
flight  parameters  get  reset  to  zero? 

B.4  REVIEW  OF  COMMON  SAFEGUARDS 

Many  UAV  designs  take  similar  approaches  ("return  home"  modes,  FTS,  parachutes,  etc.)  to 
safeguards  in  order  to  reduce  the  risk  associated  with  loss  of  control.  Some  of  these  approaches 
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have  not  always  been  adequate.  This  section  asks  questions  related  to  the  adequacy  of  those 
approaches  to  loss  of  control  safeguards,  based  on  previous  experience  with  several  UAV 
designs. 

B.4.1  Degraded  Modes  of  Flight. 

What  subsystems  will  fail  and  cause  the  UAV  not  to  be  able  to  continue  flying? 

Loss  of  which  subsystems  will  cause  the  flight  to  be  aborted  (i.e.,  precautionary  return  to  base)? 

B.4.2  Return  Home  Modes. 

Does  this  vehicle  have  an  automatic  "return  home"  feature  (also  called  "reversion  mode"  or 
"Preprogrammed  Emergency  Mission"  in  some  vehicles)  in  the  event  of  loss  of  link? 

What  conditions  cause  the  vehicle  to  go  into  "return  home"  mode? 

What  does  the  vehicle  do  once  it  arrives  at  the  "return  home"  point?  Will  it  climb  to  a  specific 
altitude?  Orbit?  Can  it  land  itself?  What  is  the  timing  and  sequence  of  events? 

B.4.2.1  Selection  of  “Return  Home”  Point. 

Is  the  selected  "return  home"  point  a  safe  place  to  bring  an  uncontrolled  vehicle? 

Can  the  "return  home"  point  be  any  location,  or  just  the  takeoff  point? 

Does  flight  path  to  “return  home  point”  from  all  points  in  the  test  flight  plan  pass  over  populated 
areas?  Will  the  vehicle  cross  any  airspace  boundaries?  Any  mountains  or  towers  higher  than  its 
altitude? 

During  "return  home"  mode,  are  altitude  limits  defined  (airspace  deconfliction  question)?  Are 
these  altitude  limits  compatible  with  the  airspace?  What  happens  if  the  altitude  limits  are 
exceeded? 

Will  the  vehicle  be  high  enough  and/or  close  enough  to  be  in  line  of  sight  of  primary  and  backup 
ground  stations? 

Are  there  multiple  “return  home”  points? 

B.4.2.2  Operator  Entry  of  "Return  Home*1  Mode  Position. 

How  is  the  “return  home”  position  entered? 

What  safeguards  prevent  erroneous  position  input? 
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If  the  UAV  is  required  to  go  to  an  intermediate  waypoint  before  the  "return  home"  point,  how  is 
the  waypoint  entered  and  how  is  it  verified? 

Is  there  a  pre-launch  check  of  the  "return  home"  mode?  Can  the  "return  home"  mode  "fly  to" 
position  be  corrected  or  updated  in  flight? 

B.4.2.3  GPS  Vs  Dead  Reckoning  (DR)  Navigation  Source  and  "Return  Home"  Mode. 

How  does  "return  home"  mode  navigate  (dead  reckoning,  inertial  nav,  radio  beacon  homing, 
GPS)? 

Is  the  reversionary  mode  tied  to  GPS?  What  happens  if  GPS  is  not  being  received  or  GPS 
jamming  tests  are  being  conducted? 

Is  there  a  DR  (dead  reckoning)  "return  home"  mode  if  GPS  or  inertial  driven  navigation  is 
unavailable  or  degraded? 

B.4.2.4  Failure  to  Regain  Control. 

What  happens  if  the  UAV  operator  fails  to  regain  control  of  the  vehicle  once  it  arrives  at  the 
"return  home"  point  and  climbs  to  altitude?  Is  there  a  time  limit?  Does  a  “Fail  Safe”  event 
occur?  Does  it  try  to  land? 

B.4.3  Ditching/Dead  Stick  Landings. 

What  situations  would  cause  the  UAV  operator  to  perform  a  forced  landing? 

B.4.3.1  Pre-planned  Ditching  Locations. 

Do  pre-planned  ditching  or  forced  landing  locations  exist?  Can  these  locations  be  reached  from 
any  point  in  the  planned  route  of  flight? 

What  is  the  criteria  for  the  selection  of  those  locations? 

How  do  you  know  if  these  locations  will  be  clear  of  people?  Will  the  locations  be  in  a  controlled 
area  or  under  surveillance? 

B.4.4  FLIGHT  TERMINATION  SYSTEM 

B.4.4.1  FTS  Function. 

Is  a  flight  termination  system  (FTS)  installed?  What  hazards  does  it  address? 

What  happens  if  the  UAV  is  below  the  RF  horizon  for  both  FTS  transmitter  and  vehicle 
command  and  control  links? 
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What  happens  when  the  FTS  activates?  Shut  off  propulsion?  Tumble  or  glide?  Does  it  deploy  a 
parachute? 

Who  has  FTS  activation  command  authority?  Vehicle  operator?  Mission  commander?  Range 
safety? 

How  are  vehicle  termination  parameters  monitored? 

B.4.4.2  FTS  Transmitter. 

Where  is  the  FTS  transmitter  located? 

Does  FTS  coverage  equal  or  exceed  the  command  transmitter  coverage?  Does  the  coverage  meet 
or  exceed  the  maximum  range  the  UAV  will  fly? 

B.4.4.3  Flight  Termination  Criteria. 

What  is  the  criteria  for  command  activation  of  the  FTS?  Does  the  criteria  include: 

•  Loss  of  all  tracking  data 

•  After  all  other  remedial  actions  have  been  taken,  a  vehicle  that  cannot  be  contained 
within  the  operating  area  or  range 

•  If  during  loss  of  link  mode,  a  vehicle  that  does  not  fly  to  a  predetermined  “return 
home"  point 

Is  the  FTS  activation  criteria  adequate  to  ensure  a  "good"  vehicle  is  not  interpreted  as  "bad, 
causing  inappropriate  use  of  the  FTS? 
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B.4.4.4  FTS  Testing  and  Certification. 


Who  certifies  the  FTS  as  "flight  ready,"  and  what  processes  are  involved  in  issuing  the 
certification? 

Is  the  flight  termination  system  independent  of  other  vehicle  systems?  Does  it  have  its  own 
antenna,  receiver,  signal  processing  capability,  and  power  supply? 

B.4.5  Fail  Safe  Mode 

Is  there  a  “fail  safe”  mode  that  comes  into  play  if  FTS  command  is  not  received?  What 
conditions  cause  it  to  activate?  What  happens  (engine  shut  off,  flight  controls  to  “turn”  or 
“tumble”)? 

What  causes  self  activation  of  the  flight  termination  system?  Electrical  power  loss?  Loss  of 
flight  critical  function?  Loss  of  FTS  signal? 

Is  there  a  specified  time  delay  between  what  triggers  fail  safe  mode  and  actions  taken  to  cause 
the  vehicle  to  stop  flying? 

B.4.6  Parachute. 

If  the  UAV  has  a  parachute  system,  at  what  altitude  will  the  chute  deploy  and  what  is  the  impact 
and  drift  rate? 

What  is  the  rate  of  descent  at  max  weight? 

Are  there  altitude,  airspeed,  or  attitude  limits  on  deploying  the  parachute? 

Does  the  UAV  have  a  weight-on-gear  inhibit  for  the  parachute  system?  How  is  it  tested  and  is 
the  status  sent  back  to  the  ground  with  telemetry? 

Does  the  engine  have  to  shut  off  prior  to  the  deployment  of  the  parachute,  and  what  happens  if 
the  engine  fails  to  shutdown?  Can  the  propeller  cut  the  parachute  shroud  line? 

B.5  QUESTIONS  ABOUT  “MIDAIR  COLLISION”  HAZARDS 

B.5.1  Airspace. 

Will  test  procedures  require  exclusive  airspace?  If  not,  how  will  risk  to  other  aircraft  be 
minimized? 

If  shared,  is  UAV  airspace  use  compatible  or  incompatible  with  any  type  aircraft  or  type 
mission? 
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How  will  air  traffic  control  occur  with  a  UAV  in  the  same  airspace  as  manned  aircraft? 


B.5.2  UAV  Routes. 

Do  planned  test  routes  consider  locations  of  published  standard  approaches  and  departures? 

Does  the  test  plan  specify  standoff  distances  from  densely  populated  areas 
(schools/hospitals/nursing  homes)?  Are  those  sites  identified? 

Are  standoffs  required  for  hazardous  sites  (fuel  depots,  weapons  storage,  etc.)? 

Does  the  test  plan  address  standoff  distances  from  small  civilian  airfields? 

Do  "return  home"  mode  locations  account  for  standoffs? 

B.5.3  Collision  Avoidance. 

How  does  the  UAV  operator  “see  and  avoid”  other  aircraft  that  may  be  nearby  (radar,  IFF, 
visual)? 

What  does  the  vehicle  use  to  ensure  pilots  of  other  aircraft  will  see  it  (TCAS,  strobes,  bright 
paint  scheme)? 

B.5.3. 1  Chase  Aircraft. 

If  a  chase  aircraft  is  used  to  help  ensure  collision  avoidance,  is  adequate  standoff  distance 
specified?  Can  chase  pilot  maintain  continuous  surveillance? 

What  communications  provisions  are  in  place  between  chase  pilot,  UAV  operator,  and  range 
safety? 

What  is  the  procedure  if  the  chase  pilot  loses  visual  contact  with  the  UAV? 

B.5.4  Interaction  with  Air  Traffic  Control. 

Is  there  an  existing  UAV  /  ATC  memorandum  of  agreement? 

Will  ATC  be  briefed  for  this  test  or  series  of  tests?  What  is  included  in  the  brief? 

Is  there  an  explicit  communication  link  between  the  UAV  ground  control  and  ATC?  Is  there  a 
backup  link  in  case  of  emergency? 

What  are  ATC  procedures  if  an  unauthorized  aircraft  enters  exclusive  airspace  being  used  by  a 
UAV? 
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What  are  ATC  procedures  if  UAV  leaves  exclusive  airspace?  Does  ATC  monitor  for  this? 

How  do  civilian  airports  and  civilian  aircraft  corridors  affect  airspace  use  by  UAVs? 

What  are  the  weather  minimums  for  this  type  vehicle?  Can  the  UAV  fly  in  clouds  or  IFR 
conditions? 

There  may  already  be  as  much  as  a  30  second  delay  for  control  actions  between  ATC  and 
manned  aircraft.  How  much  will  this  delay  be  increased  with  the  operation  of  this  UAV? 

What  is  the  procedure  for  "loss  of  IFF"?  How  will  the  UAV  operator  recognize  that  IFF  is  not 
working?  Will  the  UAV  return  to  base  or  continue  its  mission? 
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APPENDIX  C:  PROCESS  DIAGRAMS 

C 1  Determine  if  the  UAV  is  safe  to  fly  on  this  range 

C2  Determine  adequacy  of  UAV  risk  management  program 

C3  Determine  if  casualty  expectation  risk  is  acceptable 

C4  Determine  if  risk  to  property  is  acceptable 

C5  Determine  if  midair  collision  risk  is  acceptable  (Exclusive  Use) 

C6  Determine  if  midair  collision  risk  is  acceptable  (Shared  Use) 

C7  Determine  if  midair  collision  risk  is  acceptable  (National  Airspace  System) 

C8  Determine  adequacy  of  hardware  safeguards 
C9  Determine  adequacy  of  software  safeguards 
CIO  Determine  adequacy  of  procedural  safeguards 


C-l 


Preceding  Process 


Next  Process 


UAV  RjANGE  PROPOSAL 


Cl  DETERMINE  IF  UAV  IS 
SAFE  TO  FLY  ON  THIS  RANGE 


RANGE  COMMANDER'S 
RISKDECISION 


Inputs  (Suppliers) 

UAV  Range  User: 

•  UAV  documentation 

■  Route  of  flight 

•  Casualty  expectation  analysis 

■  Vulnerable  property  to  avoid 

•  Collision  avoidance  plans 

•  Reliability  data  for  critical 
safety  systems 

•Safety  procedure  documentation 

Entry  Criteria 

•  First  flight  of  UAV  on  this 
range 

«  New  type  of  mission  or  test 

■Review  of  existing  procedures 

■  At  Range  Commander's 
discretion 


Purpose: 

To  determine  if  a  particular  UAV.  UAV  test,  or  UAV 
operation  is  safe  to  fly  on  this  range. 

Primary  Sub-processes 

■Adequacy  of  UAV  system  risk  management 

•Ca  su  alty  exp  ectatio  n 

•Property  damage 

•Midair  collision  avoidance 
•Restricted  area 

■Within  other  special  use  airspace 
•In  the  National  Airspace  System 

•Adequacy  of  safeguards 
•Hardware 
■Software 
•Pro  cedures 


AcjentS  Reviewers: 

■  Aviation  Safety. 

•  Range  Safety, 

■  System  Safety. 

•  UAV  Range  User, 

■  Airspace  Manager 


•UAV  pro  gram 
documentation 


Outputs  (Customers) 

UAV  proposal  meets  all  range 
safety  criteria 

■  All  criteria  met.  or 

■  Not  required  to  meet  criteria 


UAV  proposal  does  not  meet 
ail  range  safety  criteria 

■  Deficiencies  noted 

■  Possible  alternatives  listed 


Exit  Criteria 

■  UAV  proposal  documentation  has 
been  reviewed  and  compared  with  the 
criteria.  Recommendation  made  to 
Range  Commander. 


■  Inadequate  information  available  to 
make  decision.  Discrepancies  noted. 


Handbooks,  Standards,  Limits 

DoD  Directive  3200.11  MAJOR  RANGE  AND  TEST  FACILITY  BASE 


Metrics  and  Measurement  Criteria 

j  RCC-323-99  RANGE  SAFETY  CRITERIA  FOR  UNMANNED  AIR  VEHICLES 


Preceding  Process 
UAV  RANGE  PROPOSAL 


Inputs  (Suppliers) 

UAV  Range  User 

•Hazard  Analysis  Documentation 

■Safety  History  Data 


Entry  Criteria 

■  First  flight  of  UAV  on  this 
range 

•  New  type  of  mission  or  test 
•Review  of  existing  procedures 

•  At  Range  Commander's 
discretion 


Sub-process  of 

“Determine  if  UAV  is  Safe  to  Fly  on  this  Range" 

C2  DETERMINE  ADEQUACY  OF  - 

- >■  UAV  RISK  MANAGEMENT 

Purpose: 

Determine  if  the  UAVs  risk  management  program  is 
adequate.  Are  system  hazards  recognized  and  addressed? 
Are  risk  control  measures  available?  Has  risk  been  reduced 
to  an  acceptable  level?  _ _ _ 

Primary  Sub-processes 

•  Determine  if  review  is  required 

•Previous  reviews  may  be  adequate 

■  Verify  hazards  have  been  identified  adequately 

•  Verify  hazards  have  been  assessed 

•  Verify  risk  decisions  have  been  made  and  control 
measures  have  been  identified 

•Verify  control  measures  have  been  incorporated 

■  Review  plans  for  supervisory  follow  up  of  risk  controls 

Supporting  sub-processes 

•List  of  alternatives  if  conditions  not  met 


Agents 

Reviewers:  Aviation 
Safety. Range  Safety, 
System  Safety,  UAV 
Range  User 


RANGE  COMMANDER'S 
RISKDECISION 


Outputs  (Customers) 

Risk  Management  is 
Adequate 

•All  criteria  met,  or 
■Not  required 


Risk  Management  is 
Inadequate 

■Deficiencies  noted 
■Possible  alternatives  noted 


Exit  Criteria 

Recommendation  made  to  Range 
Commander: 

(1)  UAV  documentation  has  been 
reviewed  and  compared  with  the 
criteria,  or 

(2)  Inadequate  information  available 
to  make  a  risk  decision. 


Handbooks,  Standards,  Limits 

■  OPNAVINST  3500.39  OPERATIONAL  RISK  MANAGEMENT 

•  AFI  91-213  OPERATIONAL  RISK  MANAGEMENT  PROGRAM 

■  MIL-STD-882D  SYSTEM  SAFETY 

■  NHB  1700.7  NASA  SAFETY  POLICY  AND  REQUIREMENTS 


„  ,  emend,  ur 

•UAV  Hazard  Analyses 

(2)  Inadequate  information  available 
•Range  UAV  Safety  t0  ma|<e  a  risk  decision. 

Question  list 

Metrics  and  Measurement  Criteria 

■RCC-323-99  RANGE  SAFETY  CRITERIAFOR  UNMANNED  AIR  VEHICLES 
•Air  Force  Pamphlet  91-214  OPERATIONAL  RISK  MANAGEMENT 
IMPLEMENTATION  AND  EXECUTION 

1  Oet  2000 
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Inputs  (Suppliers) 

UAV  Range  User 
for  Containment 

■  Hazard  analysis 

■  Failure  modes 

•  Safeguards  to  ensure 
containment 

for  Equivalent  Risk 

■  Predicted  UAV  mishap  rate 

■  Proposed  route  of  flight 

•  Casualty  expectation  analysis 

Entry  Criteria 

•  First  flight  of  UAV  on  this 
range 

■  New  type  of  mission  or  test 
•Review  of  existing  procedures 


Purpose:  Determine  if  the  UAV's  risk  to  people  on  the 
ground  is  acceptable.  Can  hazardous  operation  be  contained 
in  unpopulated  area?  Or,  is  UAV  risk  to  people  on  the 
ground  no  greater  than  manned  aircraft? 


Primary  Sub-processes 

1.  Verify  hazardous  operation  can  be 
contained  within  unpopulated  areas 

OR 


Outputs  (Customers) 

Casualty  Expectation  risk  is 
acceptable 

•All  criteria  met,  or 
•Not  required 


2.  Verify  predicted  risk  to  people  on  the  ground 
is  no  greater  than  from  manned  aircraft 


Casualty  expectation  risk  is 
unacceptable  or  unknown 

•Deficiencies  noted 


Supporting  sub-processes 


•Possible  alternatives 
noted 


•Hazard  analysis 

•Casualty  expectation  calculation 

•Review  of  routing 


Agents 


Tools 


Reviewers: 


•Local  population  data 


Exit  Criteria 

Recommendation  to  Range 
Commander: 

■  Hazards  are  contained,  or 

•  Risk  is  no  more  than  manned 
aircraft ,  or 


•  At  Range  Commander's 
discretion 


■  Range  Safety, 

•  UAV  Range  User 


Handbooks,  Standards,  Limits 


•  Insufficient  data  to  make  decision 


Metrics  and  Measurement  Criteria 


■  US  Census  data.  Local  population  data  sources  (tax  maps,  etc)  •  RCC-323-99  RANGE  SAFETY  CRITERIA  FOR  UNMANNED  AIR  VEHICLES 

■  OPNAV371  D.7  GENERAL  NATOPS  ‘  R(^C  321-00  RISK  AND  LETHALITY  COMMONALITY 
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Inputs  (Suppliers) 


UAV  Range  User: 

•Route  of  flight 
•Hazard  analysis 


Range 


Purpose:  Determine  if  the  UAV's  risk  to  properly  on 
the  ground  is  acceptable. 


Primary  Sub-processes 


■Verify  route  avoids  identified  critical  properties 
along  route  of  flight 


Outputs  (Customers) 

Risk  to  High  Consequence 
property  is  acceptable. 

•Property  is  identified  and 
avoided,  or 

•Review  not  required,  no  high 
consequence  property  nearthis 
range 


•Identify  local  high 
consequence  properties 


Supporting  sub-processes 

•Local  policy  defining  critical  or  high  consequence  properties 


Risk  to  high  consequence 
property  is  unacceptable. 

•Deficiencies  noted 
•Possible  alternatives  noted 


•Critical  or  high  consequence  property  identified 


Entry  Criteria 

•First  flight  of  UAV  on  this 
range 

•New  type  of  mission  or  test 

•Review  of  existing  procedures 

•At  Range  Commander's 
discretion 


•This  criteria  complements  the  casualty  expectation  criteria 


Exit  Criteria 


Agents 

Reviewers:  Aviation 
Safety,  Range  Safety, 
System  Safety,  UAV 
Range  User 


Tools 

•Local  critical  property 


Route  chosen  to  minimize  risk  to 
critical  property. 

Or 

Insufficient  data  to  make  decision 


Handbooks,  Standards,  Limits 


Metrics  and  Measurement  Criteria 


MIL-STD-882D  STANDARD  PRACTICE  FOR  SYSTEM  SAFETY 
OPNAV371 0.7  GENERAL  NATOPS 


RCC  323-99  RANGE  SAFETY  CRITERIA  FOR  UNMANNED  AIR  VEHICLES 
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Preceding  Process 


UAV  RANGE  PROPOSAL 


Sub- process  of 

■Determine  if  UAV  is  Safe  to  Fly  on  this  Range" 

C5  DETERMINE  IF  MIDAIR  COLLISION 
RISK  IS  ACCEPTABLE  -  (  Exclusive  Use) 


RANGE  COMMANDER’S 
RISK  DECISION 


Inputs  (Suppliers) 

UAV  Range  User: 

■UAV collision  avoidance 
plans  and  procedures 

■Route  of  flight 


Entry  Criteria 

•  First  flight  of  UAVon  this  range 

■  New  type  of  mission  or  test 

•  Review  of  existing  procedures 

■  At  Range  Commander’s  discretion 


Purpose: 

Determine  if  theUAV's  risk  to  other  aircraft  is  acceptable 
if  flight  has  "exclusive  use"  of  restricted  area  or  warning  area. 

Primary  Sub-processes 

•Verify  UAV  can  be  contained  within  restricted 
airspace 

•Verify  other  aircraft  can  be  excluded  from 
restricted  airspace 

•Verify  other  airborne  participants  are  aware  of 
UAV  flight  characteristics  and  contingencies 


Supporting  sub-processes 


Agents 

Reviewers:  Aviation 
Safety. Range  Safety, 
System  Safety.  UAV 
Range  User, ATC 


Outputs  (Customers) 

UAV  collision  avoidance 
plans  and  procedures  are 
adequate 

■  All  criteria  met,  or 
•  Not  required 


UAV  collision  avoidance 
plans  and  procedures  are  not 
adequate 

•  Deficiencies  noted 

•  Possible  alternatives 
listed 

Exit  Criteria 

•  UAV  collision  avoidance  plans  and 
procedures  have  been  reviewed  and 
compared  with  the  criteria. 
Recommendation  made  to  Range 
Commander. 

■  Inadequate  information  available  to 
make  decision.  Discrepancies  noted. 


Handbooks,  Standards,  Limits 

•  Local  Air  Operations  Manual 


Metrics  and  Measurement  Criteria 

RCC  323-99  RANGE  SAFET  Y  CRITERIA  FOR  UNMANNED  AIR  VEHICLES 


Preceding  Process 


UAV  RANGE  PROPOSAL 


Inputs  (Suppliers) 

UAV  Range  User: 

•Collision  avoidance 
plans  and  procedures 

•Route  of  flight 


Sub-process  of 

■Determine  if  UAV  is  Safe  to  Fly  on  this  Range" 

C6  DETERMINE  IF  MIDAIR  COLLISION  - 

- >"  RISK  IS  ACCEPTABLE  (Shared  Use) 

P urpose :  D etermine  if  th  e  U AV’s  mida ir  collision  risk 
is  acceptable  if  flight  will  share  airspace  with  other  aircraft 
within  restricted  area  or  warning  area. 


Primary  Sub-processes 

•  Verify  UAV  can  be  contained  within  assigned 
special  use  airspace 

•  Verify  there  is  compensation  for  any  UAV 
"see  and  avoid"  limitations. 

•  Verify  there  is  consideration  of  delays  with 
ATC  instruction 


Supporting  sub-processes 


Entry  Criteria 

•First  flight  of  UAVon  this  range 

•New  type  of  mission  or  test  Agents 

Reviewers:  Aviation 

•Review  of  existing  procedures  Safety.  Range  Safety, 

System  Safety,  UAV 

•At  Range  Commander’s  discretion  Range  User,  FAA,  ATC 

Handbooks,  Standards,  Limits 

•  FAA  Handbook  7610.4  SPECIAL  MILITARY  OPERATIONS,  FAA 


Tools 


RANGE  COMMANDER'S 
RISK  DECISION 


Outputs  (Customers) 

UAV  collision  avoidance  plans 
and  procedures  are  adequate 

•  All  criteria  met,  or 

•  Not  required 


UAV  collision  avoidance  plans 
and  procedures  are  not 
adequate 

•  Deficiencies  noted 

•  Possible  alternatives 


Exit  Criteria 

■  UAV  collision  avoidance  plans  and 
procedures  have  been  reviewed  and 
compared  with  the  criteria 
Recommendation  made  to  Range 
Commander. 

•  Inadequate  information  available  to 
make  decision.  Discrepancies  noted. 


Metrics  and  Measurement  Criteria 

■RCC-323-99  RANGE  SAFETY  CRITERIA  FOR  UNMANNED  AIR  VEHICLES 
•FAA  Aviation  Regulations  (FARs) 
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Preceding  Process 


UAV  RANGE  PROPOSAL 


Sub-process  of 

"Determine  if  UAV  is  Safe  to  Fly  on  this  Range" 

C7  DETERMINE  IF  MIDAIR  COLLISION 
RISK  IS  ACCEPTABLE  FOR  FLIGHT 
_ IN  NATIONAL  AIRSPACE  SYSTE M _ 


RANGE  COMMANDER'S 
RISKDECISION 


Inputs  (Suppliers) 


UAV  Range  User: 

■Collision  avoidance 
plans  and  procedures 


Purpose:  Determine  if  the  UAV’s  midair  collision  risk  is 
acceptable  if  flight  will  enter  National  Airspace  System.  Verify 
FAAthinks  it  is  safe ,  we  agree ,  and  know  why. 


Primary  Sub-processes 


•  FAA  approval 


Outputs  (Customers) 


UAV  collision  avoidance  plans  and 
procedures  are  adequate 

■All  criteria  met,  or 


•Certificate  of 

Auth  orizatio  n  from  FAA 


Entry  Criteria 

•First  flight  of  UAV  on  this  range 
■New  type  of  mission  or  test 
•Review  of  existing  procedures 
■At  Range  Commander’s  discretion 


•Review  and  approval  by  government 
sponsor 

-  Verify  UAV  can  be  contained  within 
planned  route  of  flight 

-  Verify  there  is  compensation  for  any 
UAV  "see  and  avoid*'  limitations. 

-  Verify  there  is  consideration  of 
delays  with  ATC  instruction 


Agents 

Reviewers:  Aviation 
Safety, Range  Safety, 
System  Safety.  UAV 
Range  User,  FAA 


Tools 


UAV  collision  avoidance  plans  and 
procedures  are  not  adequate 

■Deficiencies  noted 
■  Possible  alternatives  listed 


Exit  Criteria 

•  UAV  proposal  collision  avoidance 
plans  and  procedures  have  been 
reviewed  and  compared  with  the 
criteria.  Recommendation  made  to 
Range  Commander. 

-  Inadequate  information  available  to 
make  decision.  Discrepancies  noted. 


Handbooks,  Standards,  Limits 

•  FAA  Handbook  7610. 4  SPECIAL  MILfT ARY  OPERATIONS 


Metrics  and  Measurement  Criteria 

■RCC  323-99  RANGE  SAFETY  CRITERIA  FOR  UNMANNED  AIR 
VEHICLES  10d 

•FAA  Aviation  Regulations  (FARs) 


Preceding  Process 


UAV  RANGE  PROPOSAL 


Inputs  (Suppliers) 

UAV  Range  User: 

■Description  of  hardware 
safeguards 

■Reliability  Data 

■Existing  approval  documentation 


•Determination  of  what  hardware 
safeguards  are  required  (output 
from  other  criteria) 


Entry  Criteria 

•First  flight  of  UAV  on  this  range 
•New  type  of  mission  or  test 
•Review  of  existing  procedures 
■At  Range  Commander's  discretion 


C8  DETERMINE  ADEQUACY  OF 
HARDWARE  SAFEGUARDS 


Purpose:  Determine  if  the  UAV's  hardware 
safeguards  are  adequate.  Does  the  hardware 
safeguard  address  the  requirement?  Will  it  work  if 
required? 

Primary  Sub-processes 

■Determine  what  hardware  safeguards  are  required  for 
operations  on  ( or  from)  this  range  (from  other  criteria) 
■Determine  if  the  safeguard  will  address  the  requirement. 
■Review  criteria  associated  with  this  type  safeguard.  Is  it 
appropriate  for  this  environment  and  type  of  use?  Tailor 
criteria  as  required 

■Compare  hardware  safeguard  to  tailored  criteria 

■Determine  if  there  are  deficiencies 

•Determine  possible  alternatives  if  deficiencies  exist 


Supporting  sub-processes 

■  Risk  Management  adequacy 

■  Casualty  expectation 
•Property  damage  criteria 
•Midair  collision  avoidance 


Agents 

Reviewers:  Aviation 
Safety,  Range  Safety, 
System  Safety,  UAV 
Range  User 


Tools 

•UAV  Hazard  Analyses 

•Range  UAV  Safety 
Question  list 


Outputs  (Customers) 

Safeguards  are  adequate 
•All  criteria  met,  or 
■Not  required 


Hardware  safeguards  are 
inadequate 

•Deficiencies  noted 
■Possible  alternatives  noted 


Exit  Criteria 

■  UAV  proposal  hardware  safeguard 
documentation  has  been  reviewed  and 
compared  with  the  criteria. 
Recommendation  made  to  Range 
Commander. 

■  Inadequate  information  available  to 
make  decision.  Discrepancies  noted. 


Handbooks,  Standards,  Limits 

RCC-319  Flight  Termination  Systems 

MIL-HDBK-217  Reliability  Prediction  for  Electronic  Equipment 

Reliability  standards  (various) 


Metrics  and  Measurement  Criteria 

|  RCC  323-99  RANGE  SAFETY  CRITERIA  FOR  UNMANNED  AIR  VEHICLES 


Preceding  Process 


UAV  RANGE  PROPOSAL 


Inputs  (Suppliers) 


UAV  Range  User: 

■Description  of  software 
safeguards 

•Software  safety  program 
documentation 

•Software  hazard  analyses 
■Existing  approval  documentation 


Sub- process  of 

"Determine  if  UAV  is  Safe  to  Fly  on  this  Range" 

Next  Process 

1 

C9  DETERMINE  ADEQUACY  OF 

- 

RANGE  COMMANDO'S 

H 

SOFTWARE  SAFEGUARDS 

RISKDECISION 

Purpose:  Determine  if  the  UAVs  software 
safeguards  are  adequate.  Does  the  software 
safeguard  address  the  requirement?  Will  it  work  if 
required? _ _ _ _ __ — 

Primary  Sub-processes 

•Determine  what  software  safeguards  are  required  for 
operations  on  (  or  from)  this  range 
•Determine  if  the  software  safeguards  will  adequately 
address  the  requirements. 

•Review  Software  Safety  Ftogram  Plan 
■Review  software  hazard  analyses 

•Compare  software  safeguards  to  identified  requirements 
in  hazard  analysis 

•Verify  software  safeguards  have  been  tested 

•Determine  if  there  are  deficiencies 

■Determine  possible  alternatives  if  deficiencies  exist 


Outputs  (Customers) 

Software  safeguards  are 
adequate 

•All  criteria  met,  or 
■Not  required 

Software  safeguards  are 
Inadequate 

•Deficiencies  noted 
•Possible  alternatives  noted 


Entry  Criteria 

•First  flight  of  UAV  on  this  range 
■New  type  of  mission  or  test 
•Review  of  existing  procedures 
•At  Range  Commander's  discretion 


Supporting  sub-processes 

•  Risk  Management  adequacy 

■  Casualty  expectation 

■  Property  damage  criteria 

•  Midair  collision  avoidance 


Agents 

Reviewers:  Range 
Safety.  System  Safety, 
UAV  Range  User, 
Software  Safety 


Tools 

•Software  safety 
checklists 

•Software  safety  hazard 
analyses 


Exit  Criteria 

•  UAV  proposal  software  safety 
documentation  has  been  reviewed 
and  compared  with  the  criteria. 
Recommendation  made  to  Range 
Commander. 

•  Inadequate  information  available  to 
make  decision.  Discrepancies  noted. 


Handbooks,  Standards,  Limits  Metrics  and  Measurement  Criteria 

•  Software  Safety  Handbook.  Joint  Software  Safety  Committee  Rcc  ^  RANGe  SAFETY  CRITERIA  FOR  UNMANNED  AIR  VEHICLES 


•NASA- G8- 1740. 13-96;  NASA  Guidebook  for  Safety  Critical 
Software  Analysis  and  Development 


Sub-process  of  Next  Process 

Preceding  Process  "Determine  if  UAV  is  Safe  to  Fly  on  this  Range" 


P  uroose :  D  etermine  if  the  UAV  s  procedural 

safeguards  are  adequate 

Outputs  (Customers) 

Procedural  safeguards  are 

Inputs  (Suppliers) 

UAV  Range  User: 

description  of  procedural 
safeguards 

•Existing  approval  documentation 

Reviewers: 

•Determination  of  what 

procedural  safeguards  are 

required  j 

Primary  Sub-processes 

•Determine  what  procedural  safeguards  are  required  for 

operations  on  (  or  from)  this  range 

•Determine  if  the  procedures  will  adequately  address  the 

requirements. 

•Verify  safety  related  procedures  are  in  writing,  and  have 
been  reviewed  and  approved 
•Determine  if  there  are  deficiencies 
•Determine  possible  alternatives  if  deficiencies  exist 

Supporting  sub-processes 

adequate 

■All  criteria  met,  or 

•Not  required 

Procedural  safeguards  are 
inadequate 

•Deficiencies  noted 

•Possible  alternatives  noted 

Entry  Criteria 

•First  flight  of  UAV  on  this  range 

•  Risk  Management  adequacy 

•  Casualty  expectation 

•  Property  damage  criteria 
■Midair  collision  avoidance 

Exit  Criteria 

■  UAV  proposal  safety  procedures  have 

•New  type  of  mission  or  test 

•Review  of  existing  procedures 

•At  Range  Commander’s  discretion 

Agents 

Reviewers:  Aviation 

Safety, Range  Safety, 

System  Safety,  UAV 

Range  User 

Tools 

been  reviewed  and  compared  with  the 
criteria.  Recommendation  made  to 

Range  Commander. 

•  Inadequate  information  available  to 
make  decision.  Discrepancies  noted. 

Handbooks,  Standards,  Limits 

Metrics  and  Measurement  Criteria 

RCC  323-99  RANGE  SAFETY  CRITERIA  FOR  UNMANNED  AIR  VEHICLES 
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APPENDIX  D:  CASUALTY  EXPECTATION  METHODOLOGY 


Making  an  assessment  of  casualty  expectation  is  not  an  exact  science.  The  analyst  has 
many  factors  to  consider  and  there  are  many  of  the  variables  from  case  to  case.  The  results  are 
valuable  because  they  can  help  the  decision-maker  reach  a  more  informed  decision  on  adjusting 
or  approving  a  particular  UAV  route  or  operating  area.  The  following  guidelines  are  provided 
for  the  analyst  to  consider. 

D.l  CALCULATING  CASUALTY  EXPECTATION 

Casualty  expectation  is  defined  as  the  collective  or  total  risk  to  an  exposed  population;  i.e, 
the  total  number  of  individuals  who  will  be  fatalities.  This  approach  to  estimating  casualty 
expectation  uses  the  vehicle  crash  rate,  vehicle  size,  and  local  population  density,  and  is  based  on 
the  equation: 

CE  =  PF  *  PD  *  AL  *  PK  *  S  (Dl-1) 

where  the  variable  a  defined  as 

CE  =  Casualty  Expectation 

PF  =  Probably  of  Failure  or  Mishap  per  flight  hour 
PD  =  Population  Density  per  square  mile. 

AL  =  Lethal  Area 

PK  =  Probability  of  a  Fatality  given  a  hit  (usually  assumed  to  be  1) 

S  =  Shelter  factor  (if  applicable) 

The  following  paragraphs  describe  procedures  for  addressing  each  variable. 

Casualty  Expectation  is  a  cumulative  calculation.  Therefore,  it  must  be  calculated  for  each 
segment  of  the  flight  path  and  summed  over  the  entire  flight. 

D.2  PROBABILITY  OF  FAILURE  OR  MISHAP 

The  probability  of  failure  (PF  in  equation  Dl-1)  or  mishap  is  the  expected  number  of 
mishaps  in  a  given  amount  of  time  (typically  flight  hours).  Several  options  can  be  used  to 
determine  a  mishap  rate,  based  on  the  type  and  quality  of  vehicle  history  or  reliability  data 
available,  and  accuracy  and/or  conservatism  required.  These  options  include: 

•  Actual  vehicle  mishap  data 

•  Estimates  based  on  reliability  studies 

•  Comparison  by  similarity 

•  Worst  case  assumptions 

•  A  combination  of  these  approaches 


D-l 


D.2.1  Probability  of  Failure  Based  on  Mishap  Data. 


When  available,  the  actual  vehicle  failure/mishap  rate  should  be  used.  This  computation 
requires  the  most  recent  year’s  mishap  rate  (or  average  of  last  5  years)  per  100,000  flight  hours 
and  includes  the  total  number  of  crashes  (or  failure/mishaps)  experienced  within  this  time  frame. 
Mishaps  per  1 00,000  flight  hours  is  the  typical  measure  used  for  manned  aircraft.  The  average 
probability  of  crash  can  be  calculated  directly  from  that  number.  For  example,  the  Safety  Center 
gives  a  specific  UAV’s  5  year  history  as  700  mishaps  in  100,000  flight  hours,  then  the  range 
converts  that  to  PF  =  0.007  crashes  per  flight  hour.  When  using  mishap  data,  the  range  must 
consider  the  following: 

•  The  proposed  operation  may  be  more  or  less  dangerous  than  the  type  of  operation  the 
mishap  data  is  based  on. 

•  The  mishap  data  may  be  inaccurate.  Some  UAV  programs  may  not  record  mishap 
data  or  keep  an  accurate  log  of  flight  hours. 

•  New  UAVs  may  not  have  accumulated  enough  flight  hours  to  make  an  accurate 
judgment. 

If  it  is  a  new  vehicle,  probability  of  failure  data  can  be  estimated  by  the  number  of  failures 
encountered  as  flight  hours  accumulate. 

Hours  flown  without  failure 
10 
30 
100 
300 

This  method  assumes: 

•  Stochastic  system  behavior 

•  Exponential  failure  distribution 

•  Constant  system  properties 

•  Constant  environmental  stresses 

These  properties  may  not  be  present  during  initial  test  flights  of  a  UAV. 

D.2.2  Probability  of  Failure  Based  on  Similarity. 

Mishap  data  from  similar  vehicles  might  be  considered  in  estimating  probability  of  failure 
when  adequate  data  is  not  available  on  the  actual  UAV.  An  assessment  must  be  made  of  the 
differences  between  the  baseline  vehicle  and  the  vehicle  to  be  tested,  and  whether  or  not  these 
differences  significantly  affect  flight  performance  or  controllability.  For  example,  using  Pioneer 


95%  Confidence  that  PF  is  equal  or  less  than 
3X10-1 
1  X  10-1 
3  X  10-2 
1  X  10-2 
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mishap  data  for  a  Pioneer  variant  might  be  valid;  but  using  Pioneer  data  for  a  new  VTOL  UAV 
would  be  unacceptable. 

D.2.3  Estimates  From  Reliability  Studies. 


System  safety  or  reliability  assessments  based  on  Fault  Tree  Analysis  (FT A)  or  Failure 
Mode,  Effects,  and  Criticality  Analysis  (FMECA)  are  basic  options  for  predicting  probability  of 
failure  when  actual  data  is  lacking.  Fault  trees  are  useful  for  analyzing  complex  components  and 
systems.  The  FTA  is  a  top-down  technique  that  models  failure  pathways  within  a  total  system. 
The  failures  are  tracked  from  a  predetermined  deficient  event  or  condition  to  the  failure  that  may 
be  induced.  FTAs  can  be  used  to  identify  interrelationships  within  the  vehicle  and  the  support 
systems,  and  to  identify  common  cause  failures. 

On  the  other  hand,  FMECA  can  be  used  to  analyze  a  system  or  process  to  determine  how 
reliable  the  system  and  its  components  are,  identify  potential  failure  modes,  and  determine  the 
effect  and  criticality  of  that  failure  and  how  these  factors  can  be  modified  to  avoid  failures  and 
increase  reliability.  The  FMECA  is  a  bottom-up  technique  for  tabulating  each  system  element 
that  can  fail  and  for  assessing  the  consequences  of  each  failure.  The  FMECA  is  described  in 
MIL-STD-1629,  Failure  Mode,  Effects,  and  Criticality  Analysis  (FMECA). 

D.2.4  Worst  Case  Assumptions. 

In  extreme  cases  where  failure/mishap  and  reliability  data  or  time  are  not  available  to 
perform  an  in-depth  analysis,  a  "worst  case5  approach  can  be  examined.  If  the  risk  criteria  can 
be  satisfied,  no  further  analysis  is  required.  This  approach  will  most  likely  result  in  an  overly 
conservative  estimate  of  failure,  which  may  not  matter  if  the  UAV  flight  path  is  over  an 
unpopulated  or  sparsely  populated  area. 

Examples  of  “worst  case”  assumptions  might  be: 

•  The  UAV  will  crash  once  per  flight. 

•  The  UAV  will  crash  once  per  flight  hour. 

•  The  UAV  will  crash  in  the  most  densely  populated  area 

D.3  POPULATION  DENSITY 

In  some  cases  when  dealing  with  a  small  controlled  area,  range  personnel  counting  the 
number  of  people  or  vessels  in  the  area  may  acquire  actual  data.  In  most  situations,  however, 
population  density  can  only  be  obtained  through  census  data  or  local  tax  data.  While  population 
data  is  relatively  easy  to  acquire,  there  are  problems  associated  with  such  data  that  must  be 
accounted  for.  For  example: 

•  Population  distributions  are  not  uniform,  but  the  model  assumes  they  are. 
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•  Population  data  may  be  out  of  date.  Census  data  is  taken  every  ten  years,  and  it  takes 
a  year  or  more  for  it  to  be  published.  Therefore,  the  data  must  be  corrected  for  annual 
growth  rate,  which  may  be  negative  in  some  areas. 

•  Population  may  vary  with  seasons  (i.e.,  beach  resorts). 

Alternate  sources  of  population  data  might  be  locally  available.  One  source  may  be  the 
local  tax  district.  Local  tax  maps  may  identify  occupied  structures  that  may  be  used  to  estimate 
population  distribution.  The  local  environmental  planning  office  may  also  have  population 
source  data.  As  with  census  data,  the  source,  accuracy,  and  currency  of  the  data  must  be  given 
appropriate  consideration. 

D.4  LETHAL  AREA 

Lethal  area  is  the  area  of  the  piece  of  concern  (there  may  be  multiple  pieces  if  the  vehicle 
breaks  up),  plus  a  buffer  to  account  for  the  size  of  a  person.  The  analyst  may  consider  the 
terminal  flight  path  of  the  UAV  when  determining  lethal  area.  In  some  cases,  the  analyst  may 
assume  that  the  UAV  is  gliding.  Then  the  lethal  area  footprint  is  the  swath  affected  by  the 
wingspan  and  buffer  for  the  glide  distance  of  the  last  6  feet  of  altitude,  plus  the  distance  the 
vehicle  needs  to  come  to  a  stop. 

AL  =  (L  +  2B)  *  (W  +  2B)  or  AL  =  (L  +  DG  +  DS  +  2B)  *  (W  +  2B) 

L  =  Length 
W  =  Width 

B  =  Buffer  =  1  foot  on  all  sides  (commonly  used  range  standard) 

DG  =  Glide  distance  at  6  ft  of  altitude 
DS  =  Distance  to  stop 

D.5  PROBABLY  OF  FATALITY  IF  HIT 

The  probability  of  fatality  depends  on  the  UAV’s  debris  kinetic  energy  as  shown  in  Figure 
D5-1,  taken  from  RCC  Document  321-00.  UAV  kinetic  energy  is  estimated  using  the  terminal 
velocity  or  VNE  (velocity  not  to  exceed)  for  powered  flight,  whichever  is  higher.  In  most  cases, 
and/or  to  be  conservative,  PK  is  assumed  to  be  1 ;  that  is,  any  individual  hit  by  a  UAV  is  assumed 
to  be  a  fatality.  Exceptions  might  be  for  debris  from  very  light  weight  material  UAVs. 
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Figure  D.5-1.  Probability  of  fatality  from  kinetic  energy  impact. 


The  Supplement  to  RCC  Standard  321-00,  Common  Risk  Criteria  for  National  Test 
Ranges :  Inert  Debris,  provides  the  derivation  of  this  curve 

D.6  SHELTER 

The  "shelter"  factor  variable,  as  used  in  equation  Dl-1,  is  an  estimate  of  how  exposed  a 
population  is  to  a  vehicle  or  debris  that  may  be  falling.  A  shelter  factor  of "  1 "  assumes  that  the 
entire  population  is  exposed,  and  a  shelter  factor  of  "0"  assumes  that  the  entire  population  is 
completely  sheltered.  The  shelter  variable  is  an  estimate  of  the  protection  houses,  cars,  and 
buildings  provide  and  is  based  on  how  well  those  shelters  reduce  kinetic  energy  prior  to  debris 
impacting  people. 

Some  analysts  will  use  a  shelter  factor  of  "1"  to  be  conservative.  Others  may  make 
assumptions  about  what  percentage  of  the  exposed  population  is  sheltered  by  buildings,  homes, 
cars,  boats,  or  trees.  The  Supplement  to  RCC  Standard  321-00,  Common  Risk  Criteria  for 
National  Test  Ranges:  Inert  Debris,  provides  guidance  on  the  size  and  type  of  debris  required  to 
penetrate  materials  like  wood,  fiberglass,  various  metals,  and  such  structures  as  boats,  homes, 
and  commercial  buildings. 
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